2016-01-14 17:39:55,373 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(195)) - found 1 test platforms from classpath
2016-01-14 17:39:55,405 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(244)) - adding test: cascading.BasicPipesPlatformTest, with platform: hadoop2-tez
2016-01-14 17:39:55,441 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-01-14 17:39:55,974 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-01-14 17:39:56,285 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(73)) - platform property overrides:
2016-01-14 17:39:56,532 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:39:56,534 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:39:56,647 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: B333DBFD99B146FF8ED1E3AC4904F625
2016-01-14 17:39:56,892 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.113
2016-01-14 17:39:56,927 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.149
2016-01-14 17:39:56,929 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 17:39:56,929 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 17:39:56,931 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:39:57,144 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 17:39:57,315 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-01-14 17:39:57,614 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.0.4
2016-01-14 17:39:57,617 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 17:39:57,618 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:39:57,619 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-01-14 17:39:57,620 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 17:39:57,620 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 17:39:57,621 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 17:39:57,622 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...stSimpleChain/simplechain
2016-01-14 17:39:57,622 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:39:57,643 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 17:39:57,816 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:39:57,823 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:39:57,823 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:39:57,893 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1/.tez/application_1452793197822_0001 doesn't exist and is created
2016-01-14 17:39:57,968 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:39:57,968 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:39:57,969 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1/.tez/application_1452793197822_0001
2016-01-14 17:39:58,011 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793197822_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:39:58,058 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:39:58,069 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-01-14 17:39:58,069 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-01-14 17:39:58,069 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:39:58,076 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:39:58,101 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:39:58,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:39:58,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1/.tez/application_1452793197822_0001/localmode-log-dir
2016-01-14 17:39:58,111 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1/.tez/application_1452793197822_0001/localmode-log-dir/history.txt.appattempt_1452793197822_0001_000000, maxErrors=10
2016-01-14 17:39:58,111 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:39:58,113 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793197822_0001_000000, appSubmitTime=1452793197998, launchTime=1452793198010
2016-01-14 17:39:58,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:39:58,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:39:58,123 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:39:58,123 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:39:58,170 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 17:39:58,170 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 17:39:58,170 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:39:58,184 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:42971
2016-01-14 17:39:58,214 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:39:58,270 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 17:39:58,270 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 17:39:58,270 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:39:58,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:39:58,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793197822_0001_000000, startTime=1452793198291
2016-01-14 17:39:58,292 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:39:58,371 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:39:58,389 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:39:58,389 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1452793197822_0001, dagName=[30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 17:39:58,630 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 17:39:58,721 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793197822_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1/.tez/application_1452793197822_0001/localmode-log-dir/dag_1452793197822_0001_1.dot
2016-01-14 17:39:58,725 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1/.tez/application_1452793197822_0001/localmode-log-dir/dag_1452793197822_0001_1-tez-dag.pb.txt
2016-01-14 17:39:58,813 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793197822_0001_1
2016-01-14 17:39:58,813 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:39:58,814 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 17:39:58 Running Dag: dag_1452793197822_0001_1
2016-01-14 17:39:58,837 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:39:58,839 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793197822_0001_1, submitTime=1452793198630
2016-01-14 17:39:58,869 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D0820683A2D14F96BBF4E151F041FACF
2016-01-14 17:39:58,878 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:39:58,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:39:58,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793197822_0001_1, initTime=1452793198841
2016-01-14 17:39:58,885 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793197822_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:39:58,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:39:58,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:DAG_STARTED]: dagID=dag_1452793197822_0001_1, startTime=1452793198886
2016-01-14 17:39:58,887 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:39:58,889 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793197822_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:39:58,889 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F9E3F4FA9E054A05ADA4FF2DCB30AC92 : {F9EA2CADB007487E99C2E0648D753C1E={InputName=F9EA2CADB007487E99C2E0648D753C1E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:39:58,890 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F9EA2CADB007487E99C2E0648D753C1E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:39:58,890 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92]
2016-01-14 17:39:58,901 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92]
2016-01-14 17:39:58,901 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92]
2016-01-14 17:39:58,902 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1452793197822_0001, dagName=[30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1] test/(1/1) ...stSimpleChain/simplechain
2016-01-14 17:39:58,903 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92]. Starting root input initializers: 1
2016-01-14 17:39:58,920 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:39:58,923 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F9EA2CADB007487E99C2E0648D753C1E on vertex vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92]
2016-01-14 17:39:58,930 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111]
2016-01-14 17:39:58,941 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:39:58,944 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F9EA2CADB007487E99C2E0648D753C1E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:39:58,944 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452793197822_0001, with dag id: dag_1452793197822_0001_1
2016-01-14 17:39:58,945 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:39:58,946 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:39:58,973 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:39:58,973 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111]
2016-01-14 17:39:58,987 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 17:39:58,989 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F9EA2CADB007487E99C2E0648D753C1E on vertex vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92]
2016-01-14 17:39:58,992 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111]
2016-01-14 17:39:58,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:39:58,996 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E28BDCDDF8D94E84A142431119621111, vertexId=vertex_1452793197822_0001_1_01, initRequestedTime=1452793198921, initedTime=1452793198994, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:39:58,996 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:39:58,997 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:39:58,998 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F9EA2CADB007487E99C2E0648D753C1E=forAllWorkUnits=true, update=[1]}
2016-01-14 17:39:58,999 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92] parallelism set to 1
2016-01-14 17:39:59,000 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:39:59,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793197822_0001_1_00, eventCount=1
2016-01-14 17:39:59,018 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:39:59,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9E3F4FA9E054A05ADA4FF2DCB30AC92, vertexId=vertex_1452793197822_0001_1_00, initRequestedTime=1452793198889, initedTime=1452793199018, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:39:59,019 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92] with distanceFromRoot: 0
2016-01-14 17:39:59,019 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:39:59,019 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF]
2016-01-14 17:39:59,038 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:39:59,039 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF]
2016-01-14 17:39:59,039 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF]
2016-01-14 17:39:59,040 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF]
2016-01-14 17:39:59,040 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=96E2E0BE84094B6E95BCBDC90D4C65A3, vertexId=vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:39:59,043 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=96E2E0BE84094B6E95BCBDC90D4C65A3, vertexId=vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF]
2016-01-14 17:39:59,053 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=96E2E0BE84094B6E95BCBDC90D4C65A3, vertexName=D0820683A2D14F96BBF4E151F041FACF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:39:59,060 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=96E2E0BE84094B6E95BCBDC90D4C65A3, vertexId=vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF]
2016-01-14 17:39:59,062 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:39:59,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D0820683A2D14F96BBF4E151F041FACF, vertexId=vertex_1452793197822_0001_1_02, initRequestedTime=1452793199019, initedTime=1452793199062, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:39:59,062 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:39:59,064 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F9E3F4FA9E054A05ADA4FF2DCB30AC92
2016-01-14 17:39:59,064 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92]
2016-01-14 17:39:59,066 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:39:59,066 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793197822_0001_1_00, startRequestedTime=1452793199063, startedTime=1452793199063
2016-01-14 17:39:59,067 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:39:59,090 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:39:59,091 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_STARTED]: vertexName=F9E3F4FA9E054A05ADA4FF2DCB30AC92, taskId=task_1452793197822_0001_1_00_000000, scheduledTime=1452793199090, launchTime=1452793199090
2016-01-14 17:39:59,091 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793197822_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:39:59,091 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793197822_0001_1_00 for vertex: vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111] numStartedSources: 1 numSources: 1
2016-01-14 17:39:59,092 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111] with distanceFromRoot: 1
2016-01-14 17:39:59,093 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92]
2016-01-14 17:39:59,098 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F9E3F4FA9E054A05ADA4FF2DCB30AC92 in vertex: E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,100 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E28BDCDDF8D94E84A142431119621111 with 1 source tasks and 1 pending tasks
2016-01-14 17:39:59,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:39:59,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793197822_0001_1_01, startRequestedTime=1452793199094, startedTime=1452793199094
2016-01-14 17:39:59,101 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:39:59,105 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1] test/(1/1) ...stSimpleChain/simplechain, VertexName: F9E3F4FA9E054A05ADA4FF2DCB30AC92, VertexParallelism: 1, TaskAttemptID:attempt_1452793197822_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9EA2CADB007487E99C2E0648D753C1E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E28BDCDDF8D94E84A142431119621111, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:39:59,107 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793197822_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:39:59,108 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793197822_0001_1_01 for vertex: vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF] numStartedSources: 1 numSources: 1
2016-01-14 17:39:59,108 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF] with distanceFromRoot: 2
2016-01-14 17:39:59,108 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:39:59,110 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E28BDCDDF8D94E84A142431119621111 in vertex: D0820683A2D14F96BBF4E151F041FACF
2016-01-14 17:39:59,110 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D0820683A2D14F96BBF4E151F041FACF with 1 source tasks and 1 pending tasks
2016-01-14 17:39:59,111 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:39:59,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793197822_0001_1_02, startRequestedTime=1452793199110, startedTime=1452793199110
2016-01-14 17:39:59,112 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:39:59,147 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:39:59,150 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:39:59,150 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:39:59,154 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793197822_0001_00_000001
2016-01-14 17:39:59,155 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:39:59,164 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793197822_0001_00_000001
2016-01-14 17:39:59,164 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:39:59,170 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:39:59,171 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:39:59,171 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793197822_0001_00_000001, launchTime=1452793199170
2016-01-14 17:39:59,174 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793197822_0001_00_000001
2016-01-14 17:39:59,176 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793197822_0001_1_00_000000_0] to container: [container_1452793197822_0001_00_000001]
2016-01-14 17:39:59,177 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:39:59,178 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793197822_0001_00_000001 given task: attempt_1452793197822_0001_1_00_000000_0
2016-01-14 17:39:59,178 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793197822_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793197822_0001_1_00_000000_0
2016-01-14 17:39:59 Starting to run new task attempt: attempt_1452793197822_0001_1_00_000000_0
2016-01-14 17:39:59,179 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793197822_0001_1_00_000000_0
2016-01-14 17:39:59,179 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:39:59,179 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793197822_0001_1_00_000000_0] started. Is using containerId: [container_1452793197822_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:39:59,180 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:39:59,185 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:39:59,181 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9E3F4FA9E054A05ADA4FF2DCB30AC92, taskAttemptId=attempt_1452793197822_0001_1_00_000000_0, startTime=1452793199178, containerId=container_1452793197822_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793197822_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:39:59,186 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793197822_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:39:59,186 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-01-14 17:39:59,185 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:39:59,189 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793197822_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:39:59,189 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:39:59,191 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:39:59,211 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:39:59,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1] test/(1/1) ...stSimpleChain/simplechain, VertexName: F9E3F4FA9E054A05ADA4FF2DCB30AC92, VertexParallelism: 1, TaskAttemptID:attempt_1452793197822_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9EA2CADB007487E99C2E0648D753C1E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E28BDCDDF8D94E84A142431119621111, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:39:59,214 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:39:59,219 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793197822_0001_1_00_000000_0
2016-01-14 17:39:59,220 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:39:59,220 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:39:59,225 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793197822_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:39:59,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F9EA2CADB007487E99C2E0648D753C1E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:39:59,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:39:59,230 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:39:59,231 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E28BDCDDF8D94E84A142431119621111, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:39:59,231 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F9EA2CADB007487E99C2E0648D753C1E
2016-01-14 17:39:59,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:39:59,234 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F9EA2CADB007487E99C2E0648D753C1E
2016-01-14 17:39:59,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,240 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:39:59,256 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:39:59,256 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,258 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:39:59,260 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F9EA2CADB007487E99C2E0648D753C1E
2016-01-14 17:39:59,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F9EA2CADB007487E99C2E0648D753C1E
2016-01-14 17:39:59,268 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9E3F4FA9E054A05ADA4FF2DCB30AC92, ordinal: 0
2016-01-14 17:39:59,269 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9E3F4FA9E054A05ADA4FF2DCB30AC92, mem on start (mb), free: 200, total: 281, max: 672
2016-01-14 17:39:59,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:39:59,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:39:59,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:39:59,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:39:59,271 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:39:59,279 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:39:59,280 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:39:59,280 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:39:59,280 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:39:59,281 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:39:59,282 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F9EA2CADB007487E99C2E0648D753C1E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:39:59,282 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E28BDCDDF8D94E84A142431119621111, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:39:59,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:39:59,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F9EA2CADB007487E99C2E0648D753C1E
2016-01-14 17:39:59,283 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:39:59,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F9EA2CADB007487E99C2E0648D753C1E
2016-01-14 17:39:59,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F9EA2CADB007487E99C2E0648D753C1E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:39:59,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:39:59,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:39:59,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:39:59,289 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793197822_0001_1_00_000000_0
2016-01-14 17:39:59,291 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:39:59,299 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@42e13ec6
2016-01-14 17:39:59,311 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@42e13ec6
2016-01-14 17:39:59,312 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:39:59,313 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:39:59,327 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:39:59,360 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: F9EA2CADB007487E99C2E0648D753C1E
2016-01-14 17:39:59,360 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 7D9B9878EE304C24BD13C1AF20D004E0
2016-01-14 17:39:59,361 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 7D9B9878EE304C24BD13C1AF20D004E0
2016-01-14 17:39:59,382 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@64b29995; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@98138d9; comparator=cascading.tuple.hadoop.util.TupleComparator@2bb336f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:39:59,390 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:39:59,393 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:39:59,393 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:39:59,534 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:39:59,535 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:39:59,535 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:39:59,535 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:39:59,536 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:39:59,542 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:39:59,542 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:39:59,545 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9E3F4FA9E054A05ADA4FF2DCB30AC92, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:39:59,554 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793197822_0001_1_00_000000_0
2016-01-14 17:39:59,554 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9E3F4FA9E054A05ADA4FF2DCB30AC92, mem on close (mb), free: 88, total: 281, max: 672
2016-01-14 17:39:59,554 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:39:59,554 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:39:59,555 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-01-14 17:39:59,555 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-01-14 17:39:59,569 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:39:59 Completed running task attempt: attempt_1452793197822_0001_1_00_000000_0
2016-01-14 17:39:59,589 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793197822_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:39:59,590 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793197822_0001_1_00, eventCount=1
2016-01-14 17:39:59,593 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793199545, Process_Duration=9, Process_End_Time=1452793199554, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 17:39:59,593 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:39:59,594 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:39:59,596 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9E3F4FA9E054A05ADA4FF2DCB30AC92, taskAttemptId=attempt_1452793197822_0001_1_00_000000_0, startTime=1452793199178, finishTime=1452793199590, timeTaken=412, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793199545, Process_Duration=9, Process_End_Time=1452793199554, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 17:39:59,598 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:39:59,598 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793197822_0001_00_000001
2016-01-14 17:39:59,600 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793197822_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:39:59,601 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:39:59,603 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793197822_0001_1_00_000000_0
2016-01-14 17:39:59,606 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:39:59,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_FINISHED]: vertexName=F9E3F4FA9E054A05ADA4FF2DCB30AC92, taskId=task_1452793197822_0001_1_00_000000, startTime=1452793199178, finishTime=1452793199604, timeTaken=426, status=SUCCEEDED, successfulAttemptID=attempt_1452793197822_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793199545, Process_Duration=9, Process_End_Time=1452793199554, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 17:39:59,621 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793197822_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:39:59,621 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:39:59,623 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92] : 1
2016-01-14 17:39:59,623 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92]
2016-01-14 17:39:59,630 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:39:59,635 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_FINISHED]: vertexName=F9E3F4FA9E054A05ADA4FF2DCB30AC92, vertexId=vertex_1452793197822_0001_1_00, initRequestedTime=1452793198889, initedTime=1452793199018, startRequestedTime=1452793199063, startedTime=1452793199063, finishTime=1452793199623, timeTaken=560, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=756, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793199545, Process_Duration=9, Process_End_Time=1452793199554, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452793199178, firstTasksToStart=[ task_1452793197822_0001_1_00_000000 ], lastTaskFinishTime=1452793199590, lastTasksToFinish=[ task_1452793197822_0001_1_00_000000 ], minTaskDuration=412, maxTaskDuration=412, avgTaskDuration=412.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793197822_0001_1_00_000000 ], longestDurationTasks=[ task_1452793197822_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:39:59,635 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:39:59,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:39:59,638 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793197822_0001_00_000001, stoppedTime=1452793199637, exitStatus=0
2016-01-14 17:39:59,639 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:39:59,639 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111] attempt: attempt_1452793197822_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:39:59,640 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,640 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111]
2016-01-14 17:39:59,641 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 17:39:59,642 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793197822_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:39:59,642 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:39:59,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_STARTED]: vertexName=E28BDCDDF8D94E84A142431119621111, taskId=task_1452793197822_0001_1_01_000000, scheduledTime=1452793199642, launchTime=1452793199642
2016-01-14 17:39:59,643 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793197822_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:39:59,643 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111]
2016-01-14 17:39:59,643 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:39:59,644 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1] test/(1/1) ...stSimpleChain/simplechain, VertexName: E28BDCDDF8D94E84A142431119621111, VertexParallelism: 1, TaskAttemptID:attempt_1452793197822_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9E3F4FA9E054A05ADA4FF2DCB30AC92, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D0820683A2D14F96BBF4E151F041FACF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:39:59,644 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793197822_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:39:59,644 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:39:59,645 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793197822_0001_00_000002
2016-01-14 17:39:59,645 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:39:59,648 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793197822_0001_00_000002
2016-01-14 17:39:59,648 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:39:59,649 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:39:59,649 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:39:59,650 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793197822_0001_00_000002, launchTime=1452793199649
2016-01-14 17:39:59,651 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793197822_0001_00_000002
2016-01-14 17:39:59,652 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793197822_0001_1_01_000000_0] to container: [container_1452793197822_0001_00_000002]
2016-01-14 17:39:59,652 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:39:59,652 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793197822_0001_00_000002 given task: attempt_1452793197822_0001_1_01_000000_0
2016-01-14 17:39:59,653 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793197822_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793197822_0001_1_01_000000_0
2016-01-14 17:39:59,653 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793197822_0001_1_01_000000_0] started. Is using containerId: [container_1452793197822_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:39:59 Starting to run new task attempt: attempt_1452793197822_0001_1_01_000000_0
2016-01-14 17:39:59,653 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E28BDCDDF8D94E84A142431119621111, taskAttemptId=attempt_1452793197822_0001_1_01_000000_0, startTime=1452793199653, containerId=container_1452793197822_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793197822_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:39:59,653 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793197822_0001_1_01_000000_0
2016-01-14 17:39:59,654 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793197822_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:39:59,654 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:39:59,654 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:39:59,654 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:39:59,654 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:39:59,654 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:39:59,655 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:39:59,654 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793197822_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:39:59,655 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:39:59,655 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1] test/(1/1) ...stSimpleChain/simplechain, VertexName: E28BDCDDF8D94E84A142431119621111, VertexParallelism: 1, TaskAttemptID:attempt_1452793197822_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9E3F4FA9E054A05ADA4FF2DCB30AC92, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D0820683A2D14F96BBF4E151F041FACF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:39:59,655 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:39:59,656 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793197822_0001_1_01_000000_0
2016-01-14 17:39:59,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:39:59,659 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:39:59,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F9E3F4FA9E054A05ADA4FF2DCB30AC92, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:39:59,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D0820683A2D14F96BBF4E151F041FACF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:39:59,661 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:39:59,660 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:39:59,661 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793197822_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:39:59,662 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:39:59,663 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D0820683A2D14F96BBF4E151F041FACF
2016-01-14 17:39:59,663 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F9E3F4FA9E054A05ADA4FF2DCB30AC92
2016-01-14 17:39:59,665 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793197822_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:39:59,665 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793197822_0001_00_000001: 66 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:39:59,666 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793197822_0001_00_000001, Exiting
2016-01-14 17:39:59,666 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793197822_0001_00_000001
2016-01-14 17:39:59,666 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793197822_0001_00_000001
2016-01-14 17:39:59,668 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:39:59,670 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D0820683A2D14F96BBF4E151F041FACF
2016-01-14 17:39:59,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D0820683A2D14F96BBF4E151F041FACF
2016-01-14 17:39:59,669 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793197822_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:39:59,678 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793197822_0001_00_000001 completed successfully
2016-01-14 17:39:59,679 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:39:59,670 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:39:59,688 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:39:59,688 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:39:59,688 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F9E3F4FA9E054A05ADA4FF2DCB30AC92
2016-01-14 17:39:59,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F9E3F4FA9E054A05ADA4FF2DCB30AC92
2016-01-14 17:39:59,694 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E28BDCDDF8D94E84A142431119621111, ordinal: 1
2016-01-14 17:39:59,694 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E28BDCDDF8D94E84A142431119621111, mem on start (mb), free: 77, total: 281, max: 672
2016-01-14 17:39:59,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:39:59,694 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:39:59,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:39:59,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:39:59,695 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:39:59,695 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:39:59,696 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:39:59,696 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:39:59,696 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 24, TotalRequested: 739036352, TotalRequestedScaled: 3.69518176E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:1.05
2016-01-14 17:39:59,696 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D0820683A2D14F96BBF4E151F041FACF, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=66985083
2016-01-14 17:39:59,697 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F9E3F4FA9E054A05ADA4FF2DCB30AC92, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=405125774
2016-01-14 17:39:59,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:39:59,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F9E3F4FA9E054A05ADA4FF2DCB30AC92 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:39:59,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F9E3F4FA9E054A05ADA4FF2DCB30AC92
2016-01-14 17:39:59,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:39:59,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:39:59,707 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:39:59,709 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:39:59,712 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F9E3F4FA9E054A05ADA4FF2DCB30AC92 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:39:59,714 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=405125774. Updated to: ShuffleMem=405125774, postMergeMem: 0
2016-01-14 17:39:59,714 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=405125774, maxSingleShuffleLimit=101281440, mergeThreshold=364613184, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:39:59,719 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:39:59,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F9E3F4FA9E054A05ADA4FF2DCB30AC92
2016-01-14 17:39:59,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:39:59,725 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793197822_0001_1_01_000000_0
2016-01-14 17:39:59,733 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793197822_0001_1_00_000000_0_10003, runDuration: 333299, ]
2016-01-14 17:39:59,740 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 17:39:59,751 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793197822_0001_1_00_000000_0_10003] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1452793199751, TimeTaken=2, Rate=0.10 MB/s
2016-01-14 17:39:59,752 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F9E3F4FA9E054A05ADA4FF2DCB30AC92
2016-01-14 17:39:59,753 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:39:59,753 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F9E3F4FA9E054A05ADA4FF2DCB30AC92] #1 in 5ms
2016-01-14 17:39:59,754 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 7D9B9878EE304C24BD13C1AF20D004E0
2016-01-14 17:39:59,755 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 962F9193D587406DBBBF1D62429E78EF
2016-01-14 17:39:59,755 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F9E3F4FA9E054A05ADA4FF2DCB30AC92] #1, status:false, isInterrupted:false
2016-01-14 17:39:59,757 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 962F9193D587406DBBBF1D62429E78EF
2016-01-14 17:39:59,758 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:39:59,759 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7e7ef84a; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@41df0327; comparator=cascading.tuple.hadoop.util.TupleComparator@1848173a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:39:59,760 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:39:59,760 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:39:59,760 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-01-14 17:39:59,761 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:39:59,761 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:39:59,767 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:39:59,771 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-01-14 17:39:59,771 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F9E3F4FA9E054A05ADA4FF2DCB30AC92
2016-01-14 17:39:59,829 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 16515068(66060272)
2016-01-14 17:39:59,829 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 63
2016-01-14 17:39:59,829 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 52848232
2016-01-14 17:39:59,830 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 66060288
2016-01-14 17:39:59,830 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 16515068; length = 4128768
2016-01-14 17:39:59,831 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 7D9B9878EE304C24BD13C1AF20D004E0, for 1 inputs
2016-01-14 17:39:59,829 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:39:59,832 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E28BDCDDF8D94E84A142431119621111, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:39:59,835 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@f49617e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1a9f9f09
2016-01-14 17:39:59,843 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793197822_0001_1_01_000000_0
2016-01-14 17:39:59,843 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E28BDCDDF8D94E84A142431119621111, mem on close (mb), free: 71, total: 345, max: 672
2016-01-14 17:39:59,844 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F9E3F4FA9E054A05ADA4FF2DCB30AC92
2016-01-14 17:39:59,845 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:39:59,845 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:39:59,845 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:39:59,846 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:39:59,846 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:39:59,847 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:39:59,847 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:39:59,848 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 184; bufvoid = 66060288
2016-01-14 17:39:59,848 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 16515068(66060272); kvend = 16515040(66060160); length = 29/4128768
2016-01-14 17:39:59,850 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:39:59 Completed running task attempt: attempt_1452793197822_0001_1_01_000000_0
2016-01-14 17:39:59,854 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793197822_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:39:59,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793197822_0001_1_01, eventCount=1
2016-01-14 17:39:59,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 49 [[File System Counters FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=39, LAST_EVENT_RECEIVED=39][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793199832, Process_Duration=11, Process_End_Time=1452793199843, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=2]]
2016-01-14 17:39:59,857 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:39:59,857 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:39:59,858 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:39:59,858 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793197822_0001_00_000002
2016-01-14 17:39:59,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E28BDCDDF8D94E84A142431119621111, taskAttemptId=attempt_1452793197822_0001_1_01_000000_0, startTime=1452793199653, finishTime=1452793199856, timeTaken=203, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=39, LAST_EVENT_RECEIVED=39, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793199832, Process_Duration=11, Process_End_Time=1452793199843, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=2
2016-01-14 17:39:59,863 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793197822_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:39:59,863 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:39:59,863 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793197822_0001_1_01_000000_0
2016-01-14 17:39:59,866 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:39:59,868 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_FINISHED]: vertexName=E28BDCDDF8D94E84A142431119621111, taskId=task_1452793197822_0001_1_01_000000, startTime=1452793199653, finishTime=1452793199865, timeTaken=212, status=SUCCEEDED, successfulAttemptID=attempt_1452793197822_0001_1_01_000000_0, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=39, LAST_EVENT_RECEIVED=39, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793199832, Process_Duration=11, Process_End_Time=1452793199843, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=2
2016-01-14 17:39:59,869 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793197822_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:39:59,869 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111] : 1
2016-01-14 17:39:59,870 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111]
2016-01-14 17:39:59,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:39:59,876 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_FINISHED]: vertexName=E28BDCDDF8D94E84A142431119621111, vertexId=vertex_1452793197822_0001_1_01, initRequestedTime=1452793198921, initedTime=1452793198994, startRequestedTime=1452793199094, startedTime=1452793199094, finishTime=1452793199870, timeTaken=776, status=SUCCEEDED, diagnostics=, counters=Counters: 49, File System Counters, FILE_BYTES_READ=268, FILE_BYTES_WRITTEN=242, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=18, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, OUTPUT_BYTES=184, OUTPUT_BYTES_WITH_OVERHEAD=206, OUTPUT_BYTES_PHYSICAL=210, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=62, FIRST_EVENT_RECEIVED=39, LAST_EVENT_RECEIVED=39, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793199832, Process_Duration=11, Process_End_Time=1452793199843, Read_Duration=1, Tuples_Read=10, Tuples_Written=8, Write_Duration=2, vertexStats=firstTaskStartTime=1452793199653, firstTasksToStart=[ task_1452793197822_0001_1_01_000000 ], lastTaskFinishTime=1452793199856, lastTasksToFinish=[ task_1452793197822_0001_1_01_000000 ], minTaskDuration=203, maxTaskDuration=203, avgTaskDuration=203.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793197822_0001_1_01_000000 ], longestDurationTasks=[ task_1452793197822_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:39:59,877 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:39:59,877 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:39:59,878 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:39:59,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793197822_0001_00_000002, stoppedTime=1452793199878, exitStatus=0
2016-01-14 17:39:59,879 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:39:59,879 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF] attempt: attempt_1452793197822_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:39:59,880 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D0820683A2D14F96BBF4E151F041FACF
2016-01-14 17:39:59,880 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF]
2016-01-14 17:39:59,880 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 17:39:59,881 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793197822_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:39:59,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:39:59,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_STARTED]: vertexName=D0820683A2D14F96BBF4E151F041FACF, taskId=task_1452793197822_0001_1_02_000000, scheduledTime=1452793199881, launchTime=1452793199881
2016-01-14 17:39:59,882 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793197822_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:39:59,882 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF]
2016-01-14 17:39:59,882 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:39:59,883 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1] test/(1/1) ...stSimpleChain/simplechain, VertexName: D0820683A2D14F96BBF4E151F041FACF, VertexParallelism: 1, TaskAttemptID:attempt_1452793197822_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E28BDCDDF8D94E84A142431119621111, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=96E2E0BE84094B6E95BCBDC90D4C65A3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:39:59,883 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793197822_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:39:59,883 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:39:59,884 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793197822_0001_00_000003
2016-01-14 17:39:59,884 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:39:59,886 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793197822_0001_00_000003
2016-01-14 17:39:59,887 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:39:59,889 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:39:59,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793197822_0001_00_000003, launchTime=1452793199889
2016-01-14 17:39:59,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:39:59,890 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793197822_0001_00_000003
2016-01-14 17:39:59,891 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793197822_0001_1_02_000000_0] to container: [container_1452793197822_0001_00_000003]
2016-01-14 17:39:59,891 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:39:59,891 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793197822_0001_00_000003 given task: attempt_1452793197822_0001_1_02_000000_0
2016-01-14 17:39:59,892 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793197822_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793197822_0001_1_02_000000_0
2016-01-14 17:39:59,892 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793197822_0001_1_02_000000_0] started. Is using containerId: [container_1452793197822_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 17:39:59 Starting to run new task attempt: attempt_1452793197822_0001_1_02_000000_0
2016-01-14 17:39:59,892 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793197822_0001_1_02_000000_0
2016-01-14 17:39:59,892 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:39:59,893 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:39:59,893 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:39:59,893 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:39:59,893 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:39:59,893 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:39:59,893 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1] test/(1/1) ...stSimpleChain/simplechain, VertexName: D0820683A2D14F96BBF4E151F041FACF, VertexParallelism: 1, TaskAttemptID:attempt_1452793197822_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E28BDCDDF8D94E84A142431119621111, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=96E2E0BE84094B6E95BCBDC90D4C65A3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:39:59,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:39:59,894 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:39:59,895 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793197822_0001_1_02_000000_0
2016-01-14 17:39:59,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:39:59,895 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793197822_0001_1_02_000000_0 sent events: (0-1)
2016-01-14 17:39:59,895 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793197822_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:39:59,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D0820683A2D14F96BBF4E151F041FACF, taskAttemptId=attempt_1452793197822_0001_1_02_000000_0, startTime=1452793199892, containerId=container_1452793197822_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793197822_0001_00_000003/teamcity, completedLogs=
2016-01-14 17:39:59,896 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793197822_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:39:59,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:39:59,896 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793197822_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:39:59,896 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793197822_0001_00_000002: 34 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:39:59,897 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793197822_0001_00_000002, Exiting
2016-01-14 17:39:59,897 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793197822_0001_00_000002
2016-01-14 17:39:59,897 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793197822_0001_00_000002
2016-01-14 17:39:59,899 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793197822_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:39:59,899 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793197822_0001_00_000002 completed successfully
2016-01-14 17:39:59,900 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:39:59,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E28BDCDDF8D94E84A142431119621111, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:39:59,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:39:59,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:39:59,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=96E2E0BE84094B6E95BCBDC90D4C65A3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:39:59,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:39:59,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 96E2E0BE84094B6E95BCBDC90D4C65A3
2016-01-14 17:39:59,902 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:39:59,902 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 96E2E0BE84094B6E95BCBDC90D4C65A3
2016-01-14 17:39:59,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,906 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:39:59,914 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:39:59,914 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:39:59,914 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,924 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:39:59,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 96E2E0BE84094B6E95BCBDC90D4C65A3
2016-01-14 17:39:59,924 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D0820683A2D14F96BBF4E151F041FACF, ordinal: 2
2016-01-14 17:39:59,925 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D0820683A2D14F96BBF4E151F041FACF, mem on start (mb), free: 63, total: 345, max: 672
2016-01-14 17:39:59,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:39:59,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:39:59,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:39:59,926 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:39:59,926 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:39:59,927 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:39:59,927 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:39:59,927 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:39:59,928 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:39:59,928 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:39:59,929 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 96E2E0BE84094B6E95BCBDC90D4C65A3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:39:59,929 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E28BDCDDF8D94E84A142431119621111, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:39:59,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:39:59,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E28BDCDDF8D94E84A142431119621111 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:39:59,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:39:59,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:39:59,932 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:39:59,933 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:39:59,933 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E28BDCDDF8D94E84A142431119621111 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:39:59,933 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:39:59,933 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:39:59,934 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:39:59,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:39:59,939 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793197822_0001_1_02_000000_0
2016-01-14 17:39:59,941 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793197822_0001_1_01_000000_0_10006, runDuration: 188774, ]
2016-01-14 17:39:59,943 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 17:39:59,943 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793197822_0001_1_01_000000_0_10006] to DISK_DIRECT, CompressedSize=210, DecompressedSize=206,EndTime=1452793199943, TimeTaken=2, Rate=0.10 MB/s
2016-01-14 17:39:59,944 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,944 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:39:59,945 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E28BDCDDF8D94E84A142431119621111] #2 in 4ms
2016-01-14 17:39:59,945 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 962F9193D587406DBBBF1D62429E78EF
2016-01-14 17:39:59,946 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E28BDCDDF8D94E84A142431119621111] #2, status:false, isInterrupted:false
2016-01-14 17:39:59,946 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"], id: 96E2E0BE84094B6E95BCBDC90D4C65A3
2016-01-14 17:39:59,946 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:39:59,947 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain"]
2016-01-14 17:39:59,947 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 210 bytes from disk
2016-01-14 17:39:59,948 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 962F9193D587406DBBBF1D62429E78EF, for 1 inputs
2016-01-14 17:39:59,948 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:39:59,948 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:39:59,949 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 181 bytes
2016-01-14 17:39:59,949 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,949 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D0820683A2D14F96BBF4E151F041FACF, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:39:59,949 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:39:59,950 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2152960f; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@34f9c3e0
2016-01-14 17:39:59,954 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:39:59,955 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:39:59,955 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793197822_0001_1_02_000000_0
2016-01-14 17:39:59,956 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793197822_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 17:39:59,957 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793197822_0001_1_02_000000_0
2016-01-14 17:39:59,957 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D0820683A2D14F96BBF4E151F041FACF, mem on close (mb), free: 59, total: 345, max: 672
2016-01-14 17:39:59,957 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E28BDCDDF8D94E84A142431119621111
2016-01-14 17:39:59,958 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:39:59,958 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:39:59,958 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:39:59,958 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:39:59,958 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:39:59 Completed running task attempt: attempt_1452793197822_0001_1_02_000000_0
2016-01-14 17:39:59,959 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793197822_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 17:39:59,961 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:39:59,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793199949, Process_Duration=8, Process_End_Time=1452793199957, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-01-14 17:39:59,962 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:39:59,963 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:39:59,963 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793197822_0001_00_000003
2016-01-14 17:39:59,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D0820683A2D14F96BBF4E151F041FACF, taskAttemptId=attempt_1452793197822_0001_1_02_000000_0, startTime=1452793199892, finishTime=1452793199960, timeTaken=68, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793199949, Process_Duration=8, Process_End_Time=1452793199957, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 17:39:59,964 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793197822_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:39:59,964 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:39:59,964 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793197822_0001_1_02_000000_0
2016-01-14 17:39:59,966 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:39:59,969 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:TASK_FINISHED]: vertexName=D0820683A2D14F96BBF4E151F041FACF, taskId=task_1452793197822_0001_1_02_000000, startTime=1452793199892, finishTime=1452793199965, timeTaken=73, status=SUCCEEDED, successfulAttemptID=attempt_1452793197822_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793199949, Process_Duration=8, Process_End_Time=1452793199957, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 17:39:59,969 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793197822_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:39:59,969 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF] : 1
2016-01-14 17:39:59,969 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF]
2016-01-14 17:39:59,971 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:39:59,973 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:VERTEX_FINISHED]: vertexName=D0820683A2D14F96BBF4E151F041FACF, vertexId=vertex_1452793197822_0001_1_02, initRequestedTime=1452793199019, initedTime=1452793199062, startRequestedTime=1452793199110, startedTime=1452793199110, finishTime=1452793199970, timeTaken=860, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=242, FILE_BYTES_WRITTEN=660, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=8, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=8, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=361758720, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=210, SHUFFLE_BYTES=210, SHUFFLE_BYTES_DECOMPRESSED=206, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=210, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=16, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793199949, Process_Duration=8, Process_End_Time=1452793199957, Read_Duration=1, Tuples_Read=8, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1452793199892, firstTasksToStart=[ task_1452793197822_0001_1_02_000000 ], lastTaskFinishTime=1452793199960, lastTasksToFinish=[ task_1452793197822_0001_1_02_000000 ], minTaskDuration=68, maxTaskDuration=68, avgTaskDuration=68.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793197822_0001_1_02_000000 ], longestDurationTasks=[ task_1452793197822_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:39:59,973 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:39:59,973 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:39:59,974 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:39:59,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793197822_0001_00_000003, stoppedTime=1452793199974, exitStatus=0
2016-01-14 17:39:59,975 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:39:59,975 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793197822_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 17:39:59,975 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793197822_0001_00_000003: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:39:59,976 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793197822_0001_00_000003, Exiting
2016-01-14 17:39:59,976 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793197822_0001_00_000003
2016-01-14 17:39:59,976 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793197822_0001_00_000003
2016-01-14 17:39:59,975 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 17:39:59,978 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793197822_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:39:59,978 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793197822_0001_00_000003 completed successfully
2016-01-14 17:39:59,979 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793197822_0001_1
2016-01-14 17:39:59,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793197822_0001_1
2016-01-14 17:39:59,980 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 96E2E0BE84094B6E95BCBDC90D4C65A3 for vertex: vertex_1452793197822_0001_1_02 [D0820683A2D14F96BBF4E151F041FACF]
2016-01-14 17:39:59,993 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793197822_0001_1_01 [E28BDCDDF8D94E84A142431119621111]
2016-01-14 17:39:59,993 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793197822_0001_1_00 [F9E3F4FA9E054A05ADA4FF2DCB30AC92]
2016-01-14 17:40:00,000 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793197822_0001_1, queueSize=0
2016-01-14 17:40:00,001 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:00,002 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793197822_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793197822_0001_1, startTime=1452793198886, finishTime=1452793199993, timeTaken=1107, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2716, FILE_BYTES_WRITTEN=1658, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=14, REDUCE_INPUT_RECORDS=18, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=36, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1018691584, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=26, OUTPUT_BYTES=394, OUTPUT_BYTES_WITH_OVERHEAD=406, OUTPUT_BYTES_PHYSICAL=414, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=414, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=414, SHUFFLE_BYTES_DECOMPRESSED=406, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=414, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=56, MERGE_PHASE_TIME=78, FIRST_EVENT_RECEIVED=47, LAST_EVENT_RECEIVED=47, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358379599326, Process_Duration=28, Process_End_Time=4358379599354, Read_Duration=4, Tuples_Read=28, Tuples_Written=26, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-01-14 17:40:00,003 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793197822_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:00,003 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793197822_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:00,004 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793197822_0001_00_000003
2016-01-14 17:40:00,004 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793197822_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:00 Completed Dag: dag_1452793197822_0001_1
2016-01-14 17:40:00,007 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793197822_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:00,009 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793197822_0001_1_post
2016-01-14 17:40:00,009 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:00,010 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:00,014 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...stSimpleChain/simplechain, applicationId=application_1452793197822_0001
2016-01-14 17:40:00,017 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:00,018 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:00,018 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:00,018 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:00,018 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:00,019 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:00,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:00,021 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:00,022 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:00,022 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:00,022 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:00,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:00,023 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:00,029 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30F8751197D044C49DAA43C0B03A2149/B725C99C87D340BA89095A7406A695E1/.tez/application_1452793197822_0001
2016-01-14 17:40:00,030 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:02.412
2016-01-14 17:40:00,119 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleChain/simplechain
2016-01-14 17:40:00,157 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:00,157 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:00,237 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.041
2016-01-14 17:40:00,259 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.064
2016-01-14 17:40:00,261 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:00,261 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:00,262 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:00,301 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 17:40:00,318 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count
2016-01-14 17:40:00,397 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 17:40:00,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:00,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-01-14 17:40:00,399 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 17:40:00,399 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 17:40:00,399 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 17:40:00,400 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...splatform/testCount/count
2016-01-14 17:40:00,400 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:00,417 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 17:40:00,421 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:00,422 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:00,422 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:00,432 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7/.tez/application_1452793200422_0001 doesn't exist and is created
2016-01-14 17:40:00,450 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:00,450 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:00,451 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7/.tez/application_1452793200422_0001
2016-01-14 17:40:00,453 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793200422_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:00,470 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:00,471 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:00,473 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:00,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:00,474 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7/.tez/application_1452793200422_0001/localmode-log-dir
2016-01-14 17:40:00,474 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7/.tez/application_1452793200422_0001/localmode-log-dir/history.txt.appattempt_1452793200422_0001_000000, maxErrors=10
2016-01-14 17:40:00,475 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:00,475 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793200422_0001_000000, appSubmitTime=1452793200453, launchTime=1452793200453
2016-01-14 17:40:00,481 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:00,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:00,489 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:00,489 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:00,490 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:45163
2016-01-14 17:40:00,493 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:00,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793200422_0001_000000, startTime=1452793200496
2016-01-14 17:40:00,496 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:00,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:00,550 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:00,550 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:00,550 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1452793200422_0001, dagName=[D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7] count/(1/1) ...splatform/testCount/count
2016-01-14 17:40:00,580 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7] count/(1/1) ...splatform/testCount/count
2016-01-14 17:40:00,584 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793200422_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7/.tez/application_1452793200422_0001/localmode-log-dir/dag_1452793200422_0001_1.dot
2016-01-14 17:40:00,585 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7/.tez/application_1452793200422_0001/localmode-log-dir/dag_1452793200422_0001_1-tez-dag.pb.txt
2016-01-14 17:40:00,625 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793200422_0001_1
2016-01-14 17:40:00,625 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:00,625 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7] count/(1/1) ...splatform/testCount/count
2016-01-14 17:40:00 Running Dag: dag_1452793200422_0001_1
2016-01-14 17:40:00,638 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793200422_0001_1, submitTime=1452793200581
2016-01-14 17:40:00,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:00,640 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B737D801A8EE4047B72BC9F1EF3F7572
2016-01-14 17:40:00,640 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:00,641 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793200422_0001_1, initTime=1452793200638
2016-01-14 17:40:00,641 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793200422_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:00,641 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:00,642 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:DAG_STARTED]: dagID=dag_1452793200422_0001_1, startTime=1452793200642
2016-01-14 17:40:00,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:00,643 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:00,643 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793200422_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:00,643 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C1B68E3C5C8F4778BE97E88EBB8B8A4C : {8B87E7347A9242EA90661297BBB3B57F={InputName=8B87E7347A9242EA90661297BBB3B57F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:00,643 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8B87E7347A9242EA90661297BBB3B57F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:00,644 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C]
2016-01-14 17:40:00,644 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C]
2016-01-14 17:40:00,644 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C]
2016-01-14 17:40:00,644 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C]. Starting root input initializers: 1
2016-01-14 17:40:00,644 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:00,645 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8B87E7347A9242EA90661297BBB3B57F on vertex vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C]
2016-01-14 17:40:00,645 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572]
2016-01-14 17:40:00,645 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1452793200422_0001, dagName=[D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7] count/(1/1) ...splatform/testCount/count
2016-01-14 17:40:00,651 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:40:00,652 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572]
2016-01-14 17:40:00,653 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572]
2016-01-14 17:40:00,653 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572]
2016-01-14 17:40:00,653 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2F776786A3DA47ACB8DF8625B2EBA7F1, vertexId=vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:00,653 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2F776786A3DA47ACB8DF8625B2EBA7F1, vertexId=vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572]
2016-01-14 17:40:00,660 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:00,661 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8B87E7347A9242EA90661297BBB3B57F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:00,662 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2F776786A3DA47ACB8DF8625B2EBA7F1, vertexName=B737D801A8EE4047B72BC9F1EF3F7572, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:00,663 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:00,663 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:00,666 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 17:40:00,666 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2F776786A3DA47ACB8DF8625B2EBA7F1, vertexId=vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572]
2016-01-14 17:40:00,668 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B737D801A8EE4047B72BC9F1EF3F7572, vertexId=vertex_1452793200422_0001_1_01, initRequestedTime=1452793200645, initedTime=1452793200668, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:40:00,668 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:00,668 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:40:00,670 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8B87E7347A9242EA90661297BBB3B57F on vertex vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C]
2016-01-14 17:40:00,672 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:00,672 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8B87E7347A9242EA90661297BBB3B57F=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:00,673 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C] parallelism set to 1
2016-01-14 17:40:00,674 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:00,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793200422_0001_1_00, eventCount=1
2016-01-14 17:40:00,676 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:00,672 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452793200422_0001, with dag id: dag_1452793200422_0001_1
2016-01-14 17:40:00,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C1B68E3C5C8F4778BE97E88EBB8B8A4C, vertexId=vertex_1452793200422_0001_1_00, initRequestedTime=1452793200643, initedTime=1452793200675, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:00,677 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C] with distanceFromRoot: 0
2016-01-14 17:40:00,677 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:00,678 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C1B68E3C5C8F4778BE97E88EBB8B8A4C
2016-01-14 17:40:00,678 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C]
2016-01-14 17:40:00,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793200422_0001_1_00, startRequestedTime=1452793200677, startedTime=1452793200677
2016-01-14 17:40:00,678 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:00,678 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:00,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:00,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:TASK_STARTED]: vertexName=C1B68E3C5C8F4778BE97E88EBB8B8A4C, taskId=task_1452793200422_0001_1_00_000000, scheduledTime=1452793200680, launchTime=1452793200680
2016-01-14 17:40:00,682 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793200422_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:00,682 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793200422_0001_1_00 for vertex: vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572] numStartedSources: 1 numSources: 1
2016-01-14 17:40:00,683 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572] with distanceFromRoot: 1
2016-01-14 17:40:00,684 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C]
2016-01-14 17:40:00,684 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C1B68E3C5C8F4778BE97E88EBB8B8A4C in vertex: B737D801A8EE4047B72BC9F1EF3F7572
2016-01-14 17:40:00,685 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B737D801A8EE4047B72BC9F1EF3F7572 with 1 source tasks and 1 pending tasks
2016-01-14 17:40:00,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:00,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793200422_0001_1_01, startRequestedTime=1452793200684, startedTime=1452793200684
2016-01-14 17:40:00,688 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:00,688 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7] count/(1/1) ...splatform/testCount/count, VertexName: C1B68E3C5C8F4778BE97E88EBB8B8A4C, VertexParallelism: 1, TaskAttemptID:attempt_1452793200422_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8B87E7347A9242EA90661297BBB3B57F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B737D801A8EE4047B72BC9F1EF3F7572, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:00,689 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793200422_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:00,689 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:00,690 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:00,692 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:00,692 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:00,693 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793200422_0001_00_000001
2016-01-14 17:40:00,694 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:00,697 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793200422_0001_00_000001
2016-01-14 17:40:00,697 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:00,698 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:00,698 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:00,698 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793200422_0001_00_000001, launchTime=1452793200698
2016-01-14 17:40:00,699 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793200422_0001_00_000001
2016-01-14 17:40:00,699 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793200422_0001_1_00_000000_0] to container: [container_1452793200422_0001_00_000001]
2016-01-14 17:40:00,700 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:00,700 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793200422_0001_00_000001 given task: attempt_1452793200422_0001_1_00_000000_0
2016-01-14 17:40:00,700 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793200422_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793200422_0001_1_00_000000_0
2016-01-14 17:40:00,700 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793200422_0001_1_00_000000_0] started. Is using containerId: [container_1452793200422_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:00,701 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:00,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C1B68E3C5C8F4778BE97E88EBB8B8A4C, taskAttemptId=attempt_1452793200422_0001_1_00_000000_0, startTime=1452793200700, containerId=container_1452793200422_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793200422_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:00,702 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793200422_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:00 Starting to run new task attempt: attempt_1452793200422_0001_1_00_000000_0
2016-01-14 17:40:00,702 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793200422_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:00,702 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793200422_0001_1_00_000000_0
2016-01-14 17:40:00,702 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:00,703 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:00,703 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:00,703 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:00,703 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:00,704 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:00,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7] count/(1/1) ...splatform/testCount/count, VertexName: C1B68E3C5C8F4778BE97E88EBB8B8A4C, VertexParallelism: 1, TaskAttemptID:attempt_1452793200422_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8B87E7347A9242EA90661297BBB3B57F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B737D801A8EE4047B72BC9F1EF3F7572, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:00,704 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:00,705 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793200422_0001_1_00_000000_0
2016-01-14 17:40:00,706 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:00,706 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:00,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8B87E7347A9242EA90661297BBB3B57F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:00,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B737D801A8EE4047B72BC9F1EF3F7572, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:00,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:00,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:00,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8B87E7347A9242EA90661297BBB3B57F
2016-01-14 17:40:00,710 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8B87E7347A9242EA90661297BBB3B57F
2016-01-14 17:40:00,715 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793200422_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:00,716 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:00,718 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:00,719 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8B87E7347A9242EA90661297BBB3B57F
2016-01-14 17:40:00,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8B87E7347A9242EA90661297BBB3B57F
2016-01-14 17:40:00,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:00,721 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B737D801A8EE4047B72BC9F1EF3F7572
2016-01-14 17:40:00,726 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:00,727 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B737D801A8EE4047B72BC9F1EF3F7572
2016-01-14 17:40:00,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B737D801A8EE4047B72BC9F1EF3F7572
2016-01-14 17:40:00,734 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C1B68E3C5C8F4778BE97E88EBB8B8A4C, ordinal: 0
2016-01-14 17:40:00,734 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C1B68E3C5C8F4778BE97E88EBB8B8A4C, mem on start (mb), free: 164, total: 461, max: 672
2016-01-14 17:40:00,734 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:00,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:00,736 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:00,736 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:00,736 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:00,737 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:00,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:00,738 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:00,741 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:00,741 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:00,741 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8B87E7347A9242EA90661297BBB3B57F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:00,741 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B737D801A8EE4047B72BC9F1EF3F7572, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:00,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:00,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8B87E7347A9242EA90661297BBB3B57F
2016-01-14 17:40:00,741 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:00,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8B87E7347A9242EA90661297BBB3B57F
2016-01-14 17:40:00,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8B87E7347A9242EA90661297BBB3B57F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:00,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:00,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:00,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:00,744 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793200422_0001_1_00_000000_0
2016-01-14 17:40:00,745 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:00,746 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1a00a506
2016-01-14 17:40:00,747 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1a00a506
2016-01-14 17:40:00,747 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:00,747 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:00,753 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:00,754 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 8B87E7347A9242EA90661297BBB3B57F
2016-01-14 17:40:00,755 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 0AF71B5714004837B22B465DD46001CC
2016-01-14 17:40:00,755 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 0AF71B5714004837B22B465DD46001CC
2016-01-14 17:40:00,757 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@748a3b38; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1f36154e; comparator=cascading.tuple.hadoop.util.TupleComparator@17b0c3e4; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:00,757 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:00,758 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:00,758 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:00,852 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:00,852 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:00,853 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:00,853 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:00,853 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:00,854 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:00,855 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:00,855 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C1B68E3C5C8F4778BE97E88EBB8B8A4C, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:00,860 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793200422_0001_1_00_000000_0
2016-01-14 17:40:00,860 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C1B68E3C5C8F4778BE97E88EBB8B8A4C, mem on close (mb), free: 60, total: 461, max: 672
2016-01-14 17:40:00,860 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:00,860 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:00,861 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 17:40:00,861 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:40:00,865 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:00 Completed running task attempt: attempt_1452793200422_0001_1_00_000000_0
2016-01-14 17:40:00,871 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793200422_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:00,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793200422_0001_1_00, eventCount=1
2016-01-14 17:40:00,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793200855, Process_Duration=5, Process_End_Time=1452793200860, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:40:00,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:00,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C1B68E3C5C8F4778BE97E88EBB8B8A4C, taskAttemptId=attempt_1452793200422_0001_1_00_000000_0, startTime=1452793200700, finishTime=1452793200874, timeTaken=174, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793200855, Process_Duration=5, Process_End_Time=1452793200860, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:40:00,875 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:00,876 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:00,876 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793200422_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:00,876 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:00,877 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793200422_0001_1_00_000000_0
2016-01-14 17:40:00,877 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:00,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:TASK_FINISHED]: vertexName=C1B68E3C5C8F4778BE97E88EBB8B8A4C, taskId=task_1452793200422_0001_1_00_000000, startTime=1452793200700, finishTime=1452793200877, timeTaken=177, status=SUCCEEDED, successfulAttemptID=attempt_1452793200422_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793200855, Process_Duration=5, Process_End_Time=1452793200860, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:40:00,878 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793200422_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:00,878 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C] : 1
2016-01-14 17:40:00,878 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C]
2016-01-14 17:40:00,878 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793200422_0001_00_000001
2016-01-14 17:40:00,882 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:00,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:VERTEX_FINISHED]: vertexName=C1B68E3C5C8F4778BE97E88EBB8B8A4C, vertexId=vertex_1452793200422_0001_1_00, initRequestedTime=1452793200643, initedTime=1452793200675, startRequestedTime=1452793200677, startedTime=1452793200677, finishTime=1452793200878, timeTaken=201, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793200855, Process_Duration=5, Process_End_Time=1452793200860, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452793200700, firstTasksToStart=[ task_1452793200422_0001_1_00_000000 ], lastTaskFinishTime=1452793200874, lastTasksToFinish=[ task_1452793200422_0001_1_00_000000 ], minTaskDuration=174, maxTaskDuration=174, avgTaskDuration=174.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793200422_0001_1_00_000000 ], longestDurationTasks=[ task_1452793200422_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:00,884 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:00,885 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:00,885 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572] attempt: attempt_1452793200422_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:00,886 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B737D801A8EE4047B72BC9F1EF3F7572
2016-01-14 17:40:00,886 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572]
2016-01-14 17:40:00,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:00,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793200422_0001_00_000001, stoppedTime=1452793200887, exitStatus=0
2016-01-14 17:40:00,887 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:00,888 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:00,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:00,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:TASK_STARTED]: vertexName=B737D801A8EE4047B72BC9F1EF3F7572, taskId=task_1452793200422_0001_1_01_000000, scheduledTime=1452793200888, launchTime=1452793200888
2016-01-14 17:40:00,889 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793200422_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:00,889 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793200422_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:40:00,889 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572]
2016-01-14 17:40:00,890 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:00,890 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7] count/(1/1) ...splatform/testCount/count, VertexName: B737D801A8EE4047B72BC9F1EF3F7572, VertexParallelism: 1, TaskAttemptID:attempt_1452793200422_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C1B68E3C5C8F4778BE97E88EBB8B8A4C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2F776786A3DA47ACB8DF8625B2EBA7F1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:00,890 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793200422_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:00,890 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:00,891 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793200422_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:00,892 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793200422_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:00,892 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793200422_0001_00_000002
2016-01-14 17:40:00,892 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793200422_0001_00_000001, Exiting
2016-01-14 17:40:00,892 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793200422_0001_00_000001
2016-01-14 17:40:00,892 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793200422_0001_00_000001
2016-01-14 17:40:00,892 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:00,894 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793200422_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:00,895 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793200422_0001_00_000001 completed successfully
2016-01-14 17:40:00,895 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:00,898 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793200422_0001_00_000002
2016-01-14 17:40:00,899 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:40:00,902 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:00,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793200422_0001_00_000002, launchTime=1452793200902
2016-01-14 17:40:00,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:00,904 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793200422_0001_00_000002
2016-01-14 17:40:00,905 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793200422_0001_1_01_000000_0] to container: [container_1452793200422_0001_00_000002]
2016-01-14 17:40:00,905 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:00,906 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793200422_0001_00_000002 given task: attempt_1452793200422_0001_1_01_000000_0
2016-01-14 17:40:00,906 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793200422_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793200422_0001_1_01_000000_0
2016-01-14 17:40:00,906 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793200422_0001_1_01_000000_0] started. Is using containerId: [container_1452793200422_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:40:00 Starting to run new task attempt: attempt_1452793200422_0001_1_01_000000_0
2016-01-14 17:40:00,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:00,908 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793200422_0001_1_01_000000_0
2016-01-14 17:40:00,908 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:00,908 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:00,908 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:00,908 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:00,908 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:00,908 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:00,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7] count/(1/1) ...splatform/testCount/count, VertexName: B737D801A8EE4047B72BC9F1EF3F7572, VertexParallelism: 1, TaskAttemptID:attempt_1452793200422_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C1B68E3C5C8F4778BE97E88EBB8B8A4C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2F776786A3DA47ACB8DF8625B2EBA7F1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:00,909 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:00,910 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793200422_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:40:00,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B737D801A8EE4047B72BC9F1EF3F7572, taskAttemptId=attempt_1452793200422_0001_1_01_000000_0, startTime=1452793200906, containerId=container_1452793200422_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793200422_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:40:00,911 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793200422_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:00,912 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793200422_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:00,910 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793200422_0001_1_01_000000_0
2016-01-14 17:40:00,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:00,913 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:00,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:00,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2F776786A3DA47ACB8DF8625B2EBA7F1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:00,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:00,916 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2F776786A3DA47ACB8DF8625B2EBA7F1
2016-01-14 17:40:00,916 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:00,916 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2F776786A3DA47ACB8DF8625B2EBA7F1
2016-01-14 17:40:00,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C1B68E3C5C8F4778BE97E88EBB8B8A4C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:00,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:00,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C1B68E3C5C8F4778BE97E88EBB8B8A4C
2016-01-14 17:40:00,923 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:00,924 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:00,924 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C1B68E3C5C8F4778BE97E88EBB8B8A4C
2016-01-14 17:40:00,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C1B68E3C5C8F4778BE97E88EBB8B8A4C
2016-01-14 17:40:00,928 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:00,938 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:00,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2F776786A3DA47ACB8DF8625B2EBA7F1
2016-01-14 17:40:00,943 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B737D801A8EE4047B72BC9F1EF3F7572, ordinal: 1
2016-01-14 17:40:00,943 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B737D801A8EE4047B72BC9F1EF3F7572, mem on start (mb), free: 50, total: 461, max: 672
2016-01-14 17:40:00,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:00,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:00,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:00,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:00,944 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:00,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:00,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:00,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:00,946 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:00,946 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:40:00,946 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2F776786A3DA47ACB8DF8625B2EBA7F1, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:00,947 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C1B68E3C5C8F4778BE97E88EBB8B8A4C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:40:00,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:00,947 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C1B68E3C5C8F4778BE97E88EBB8B8A4C
2016-01-14 17:40:00,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C1B68E3C5C8F4778BE97E88EBB8B8A4C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:00,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:00,949 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:00,949 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:00,950 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:00,951 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C1B68E3C5C8F4778BE97E88EBB8B8A4C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:00,951 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:40:00,951 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:00,952 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:00,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C1B68E3C5C8F4778BE97E88EBB8B8A4C
2016-01-14 17:40:00,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:00,957 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793200422_0001_1_01_000000_0
2016-01-14 17:40:00,957 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793200422_0001_1_00_000000_0_10012, runDuration: 145407, ]
2016-01-14 17:40:00,962 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793200422_0001_1_00_000000_0_10012] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452793200962, TimeTaken=1, Rate=0.44 MB/s
2016-01-14 17:40:00,963 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C1B68E3C5C8F4778BE97E88EBB8B8A4C
2016-01-14 17:40:00,963 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:00,963 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C1B68E3C5C8F4778BE97E88EBB8B8A4C] #3 in 2ms
2016-01-14 17:40:00,964 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C1B68E3C5C8F4778BE97E88EBB8B8A4C] #3, status:false, isInterrupted:false
2016-01-14 17:40:00,965 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 17:40:00,966 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:00,967 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 0AF71B5714004837B22B465DD46001CC
2016-01-14 17:40:00,967 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 17:40:00,967 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:00,967 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"], id: 2F776786A3DA47ACB8DF8625B2EBA7F1
2016-01-14 17:40:00,967 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:00,967 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount/count"]
2016-01-14 17:40:00,969 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 0AF71B5714004837B22B465DD46001CC, for 1 inputs
2016-01-14 17:40:00,969 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 17:40:00,969 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C1B68E3C5C8F4778BE97E88EBB8B8A4C
2016-01-14 17:40:00,969 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B737D801A8EE4047B72BC9F1EF3F7572, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:00,969 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:00,971 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@78ce9fda; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4560df2c
2016-01-14 17:40:00,975 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:00,975 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:00,976 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793200422_0001_1_01_000000_0
2016-01-14 17:40:00,977 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793200422_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:40:00,978 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793200422_0001_1_01_000000_0
2016-01-14 17:40:00,978 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B737D801A8EE4047B72BC9F1EF3F7572, mem on close (mb), free: 44, total: 461, max: 672
2016-01-14 17:40:00,978 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C1B68E3C5C8F4778BE97E88EBB8B8A4C
2016-01-14 17:40:00,979 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:00,979 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:00,980 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:00,980 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:00,980 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:00 Completed running task attempt: attempt_1452793200422_0001_1_01_000000_0
2016-01-14 17:40:00,981 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793200422_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:00,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:00,983 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=589, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793200969, Process_Duration=9, Process_End_Time=1452793200978, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 17:40:00,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B737D801A8EE4047B72BC9F1EF3F7572, taskAttemptId=attempt_1452793200422_0001_1_01_000000_0, startTime=1452793200906, finishTime=1452793200983, timeTaken=77, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=589, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793200969, Process_Duration=9, Process_End_Time=1452793200978, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 17:40:00,984 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793200422_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:00,984 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793200422_0001_1_01_000000_0
2016-01-14 17:40:00,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:00,985 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:TASK_FINISHED]: vertexName=B737D801A8EE4047B72BC9F1EF3F7572, taskId=task_1452793200422_0001_1_01_000000, startTime=1452793200906, finishTime=1452793200984, timeTaken=78, status=SUCCEEDED, successfulAttemptID=attempt_1452793200422_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=589, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793200969, Process_Duration=9, Process_End_Time=1452793200978, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 17:40:00,985 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:00,986 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:00,985 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:00,987 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793200422_0001_00_000002
2016-01-14 17:40:00,986 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793200422_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:00,987 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572] : 1
2016-01-14 17:40:00,987 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572]
2016-01-14 17:40:00,988 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:00,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:VERTEX_FINISHED]: vertexName=B737D801A8EE4047B72BC9F1EF3F7572, vertexId=vertex_1452793200422_0001_1_01, initRequestedTime=1452793200645, initedTime=1452793200668, startRequestedTime=1452793200684, startedTime=1452793200684, finishTime=1452793200988, timeTaken=304, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=589, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793200969, Process_Duration=9, Process_End_Time=1452793200978, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452793200906, firstTasksToStart=[ task_1452793200422_0001_1_01_000000 ], lastTaskFinishTime=1452793200983, lastTasksToFinish=[ task_1452793200422_0001_1_01_000000 ], minTaskDuration=77, maxTaskDuration=77, avgTaskDuration=77.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793200422_0001_1_01_000000 ], longestDurationTasks=[ task_1452793200422_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:00,989 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:00,991 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:00,991 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793200422_0001_00_000002, stoppedTime=1452793200991, exitStatus=0
2016-01-14 17:40:00,991 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:00,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:00,992 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:00,992 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793200422_0001_1
2016-01-14 17:40:00,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793200422_0001_1
2016-01-14 17:40:00,993 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2F776786A3DA47ACB8DF8625B2EBA7F1 for vertex: vertex_1452793200422_0001_1_01 [B737D801A8EE4047B72BC9F1EF3F7572]
2016-01-14 17:40:00,998 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793200422_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:40:00,998 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793200422_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:00,998 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793200422_0001_00_000002, Exiting
2016-01-14 17:40:00,998 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793200422_0001_00_000002
2016-01-14 17:40:00,998 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793200422_0001_00_000002
2016-01-14 17:40:00,999 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793200422_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:01,000 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793200422_0001_00_000002 completed successfully
2016-01-14 17:40:01,023 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793200422_0001_1_00 [C1B68E3C5C8F4778BE97E88EBB8B8A4C]
2016-01-14 17:40:01,028 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793200422_0001_1, queueSize=0
2016-01-14 17:40:01,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:01,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793200422_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793200422_0001_1, startTime=1452793200642, finishTime=1452793201023, timeTaken=381, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1084, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=966787072, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905586401824, Process_Duration=14, Process_End_Time=2905586401838, Read_Duration=3, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 17:40:01,035 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793200422_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:01,035 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793200422_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:01,035 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793200422_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:01 Completed Dag: dag_1452793200422_0001_1
2016-01-14 17:40:01,036 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793200422_0001_00_000002
2016-01-14 17:40:01,036 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793200422_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:01,036 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793200422_0001_1_post
2016-01-14 17:40:01,037 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:01,037 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:01,045 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...splatform/testCount/count, applicationId=application_1452793200422_0001
2016-01-14 17:40:01,048 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:01,048 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:01,049 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:01,049 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:01,049 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:01,049 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:01,051 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:01,051 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:01,051 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:01,052 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:01,053 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:01,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:01,055 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:01,061 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D1FB5B459CEE416A9B421A326D4F931F/D4167344936C45F089C98C576A1E02A7/.tez/application_1452793200422_0001
2016-01-14 17:40:01,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.665
2016-01-14 17:40:01,148 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount/count
2016-01-14 17:40:01,182 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:01,182 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:01,238 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 17:40:01,249 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-01-14 17:40:01,250 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 17:40:01,250 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 17:40:01,250 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:01,270 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 17:40:01,271 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap
2016-01-14 17:40:01,283 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 17:40:01,283 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:01,284 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-01-14 17:40:01,284 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 17:40:01,284 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 17:40:01,284 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 17:40:01,285 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...pesplatform/testSwap/swap
2016-01-14 17:40:01,285 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:01,301 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 17:40:01,306 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:01,307 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:01,307 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:01,317 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0/.tez/application_1452793201307_0001 doesn't exist and is created
2016-01-14 17:40:01,333 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:01,333 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:01,334 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0/.tez/application_1452793201307_0001
2016-01-14 17:40:01,337 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793201307_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:01,352 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:01,353 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:01,355 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:01,355 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:01,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0/.tez/application_1452793201307_0001/localmode-log-dir
2016-01-14 17:40:01,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0/.tez/application_1452793201307_0001/localmode-log-dir/history.txt.appattempt_1452793201307_0001_000000, maxErrors=10
2016-01-14 17:40:01,356 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:01,357 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793201307_0001_000000, appSubmitTime=1452793201337, launchTime=1452793201337
2016-01-14 17:40:01,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:01,368 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:53599
2016-01-14 17:40:01,369 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:01,369 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:01,369 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:01,372 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:01,374 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:01,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793201307_0001_000000, startTime=1452793201374
2016-01-14 17:40:01,375 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:01,434 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:01,434 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:01,434 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1452793201307_0001, dagName=[9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 17:40:01,455 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 17:40:01,457 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793201307_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0/.tez/application_1452793201307_0001/localmode-log-dir/dag_1452793201307_0001_1.dot
2016-01-14 17:40:01,458 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0/.tez/application_1452793201307_0001/localmode-log-dir/dag_1452793201307_0001_1-tez-dag.pb.txt
2016-01-14 17:40:01,491 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793201307_0001_1
2016-01-14 17:40:01,491 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:01,491 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 17:40:01 Running Dag: dag_1452793201307_0001_1
2016-01-14 17:40:01,505 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793201307_0001_1, submitTime=1452793201455
2016-01-14 17:40:01,505 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:01,505 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EB03C3AB040044CA8F3557691A844B1B
2016-01-14 17:40:01,506 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:01,507 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793201307_0001_1, initTime=1452793201505
2016-01-14 17:40:01,507 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:01,508 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793201307_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:01,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:DAG_STARTED]: dagID=dag_1452793201307_0001_1, startTime=1452793201509
2016-01-14 17:40:01,509 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:01,509 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:01,510 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793201307_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:01,510 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EB03C3AB040044CA8F3557691A844B1B : {0C959E3FFCA1491C9744AD171AD858D8={InputName=0C959E3FFCA1491C9744AD171AD858D8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:01,510 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0C959E3FFCA1491C9744AD171AD858D8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:01,510 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,510 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,510 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,511 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]. Starting root input initializers: 1
2016-01-14 17:40:01,511 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1452793201307_0001, dagName=[9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0] test/(1/1) ...pesplatform/testSwap/swap
2016-01-14 17:40:01,513 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:01,513 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0C959E3FFCA1491C9744AD171AD858D8 on vertex vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,518 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:01,519 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0C959E3FFCA1491C9744AD171AD858D8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:01,520 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:01,521 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:01,524 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 17:40:01,526 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0C959E3FFCA1491C9744AD171AD858D8 on vertex vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,526 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:01,526 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0C959E3FFCA1491C9744AD171AD858D8=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:01,526 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B] parallelism set to 1
2016-01-14 17:40:01,526 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:01,527 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793201307_0001_1_00, eventCount=1
2016-01-14 17:40:01,527 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,527 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=944E19AF285E48E29F988F286002E0FE, vertexId=vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:01,527 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=944E19AF285E48E29F988F286002E0FE, vertexId=vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,529 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=944E19AF285E48E29F988F286002E0FE, vertexName=EB03C3AB040044CA8F3557691A844B1B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:01,530 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=944E19AF285E48E29F988F286002E0FE, vertexId=vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,535 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EB03C3AB040044CA8F3557691A844B1B, vertexId=vertex_1452793201307_0001_1_00, initRequestedTime=1452793201510, initedTime=1452793201535, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:01,535 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:01,536 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B] with distanceFromRoot: 0
2016-01-14 17:40:01,538 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:01,538 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EB03C3AB040044CA8F3557691A844B1B
2016-01-14 17:40:01,539 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,539 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452793201307_0001, with dag id: dag_1452793201307_0001_1
2016-01-14 17:40:01,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793201307_0001_1_00, startRequestedTime=1452793201538, startedTime=1452793201538
2016-01-14 17:40:01,539 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:01,541 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:01,543 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:01,543 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:TASK_STARTED]: vertexName=EB03C3AB040044CA8F3557691A844B1B, taskId=task_1452793201307_0001_1_00_000000, scheduledTime=1452793201542, launchTime=1452793201542
2016-01-14 17:40:01,543 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793201307_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:01,543 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,544 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0] test/(1/1) ...pesplatform/testSwap/swap, VertexName: EB03C3AB040044CA8F3557691A844B1B, VertexParallelism: 1, TaskAttemptID:attempt_1452793201307_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0C959E3FFCA1491C9744AD171AD858D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=944E19AF285E48E29F988F286002E0FE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:01,544 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793201307_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:01,544 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:01,546 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:01,547 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:01,547 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:01,548 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793201307_0001_00_000001
2016-01-14 17:40:01,548 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201307_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:01,550 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793201307_0001_00_000001
2016-01-14 17:40:01,550 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:01,551 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201307_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:01,551 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:01,551 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793201307_0001_00_000001, launchTime=1452793201551
2016-01-14 17:40:01,552 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793201307_0001_00_000001
2016-01-14 17:40:01,552 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793201307_0001_1_00_000000_0] to container: [container_1452793201307_0001_00_000001]
2016-01-14 17:40:01,553 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201307_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:01,553 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793201307_0001_1_00_000000_0] started. Is using containerId: [container_1452793201307_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:01,553 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:01,553 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EB03C3AB040044CA8F3557691A844B1B, taskAttemptId=attempt_1452793201307_0001_1_00_000000_0, startTime=1452793201553, containerId=container_1452793201307_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793201307_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:01,554 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793201307_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:01,554 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793201307_0001_00_000001 given task: attempt_1452793201307_0001_1_00_000000_0
2016-01-14 17:40:01,554 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793201307_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:01,554 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793201307_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793201307_0001_1_00_000000_0
2016-01-14 17:40:01 Starting to run new task attempt: attempt_1452793201307_0001_1_00_000000_0
2016-01-14 17:40:01,555 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793201307_0001_1_00_000000_0
2016-01-14 17:40:01,555 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:01,556 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:01,556 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:01,556 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:01,556 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:01,557 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:01,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0] test/(1/1) ...pesplatform/testSwap/swap, VertexName: EB03C3AB040044CA8F3557691A844B1B, VertexParallelism: 1, TaskAttemptID:attempt_1452793201307_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0C959E3FFCA1491C9744AD171AD858D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=944E19AF285E48E29F988F286002E0FE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:01,557 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:01,558 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793201307_0001_1_00_000000_0
2016-01-14 17:40:01,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:01,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:01,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0C959E3FFCA1491C9744AD171AD858D8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:01,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:01,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:01,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0C959E3FFCA1491C9744AD171AD858D8
2016-01-14 17:40:01,562 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0C959E3FFCA1491C9744AD171AD858D8
2016-01-14 17:40:01,566 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:01,566 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=944E19AF285E48E29F988F286002E0FE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:01,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:01,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 944E19AF285E48E29F988F286002E0FE
2016-01-14 17:40:01,571 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:01,571 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 944E19AF285E48E29F988F286002E0FE
2016-01-14 17:40:01,568 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:01,569 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793201307_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:01,575 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0C959E3FFCA1491C9744AD171AD858D8
2016-01-14 17:40:01,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0C959E3FFCA1491C9744AD171AD858D8
2016-01-14 17:40:01,578 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EB03C3AB040044CA8F3557691A844B1B, ordinal: 0
2016-01-14 17:40:01,578 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EB03C3AB040044CA8F3557691A844B1B, mem on start (mb), free: 173, total: 461, max: 672
2016-01-14 17:40:01,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:01,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:01,579 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:01,582 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:01,582 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 944E19AF285E48E29F988F286002E0FE
2016-01-14 17:40:01,582 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:01,582 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:01,583 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:01,583 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:01,583 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:01,584 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:01,584 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:01,584 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 17:40:01,585 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0C959E3FFCA1491C9744AD171AD858D8, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:01,585 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 944E19AF285E48E29F988F286002E0FE, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:01,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:01,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0C959E3FFCA1491C9744AD171AD858D8 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:01,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0C959E3FFCA1491C9744AD171AD858D8
2016-01-14 17:40:01,586 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:01,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0C959E3FFCA1491C9744AD171AD858D8
2016-01-14 17:40:01,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:01,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:01,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:01,587 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793201307_0001_1_00_000000_0
2016-01-14 17:40:01,587 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:01,588 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@36bc5208
2016-01-14 17:40:01,589 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@36bc5208
2016-01-14 17:40:01,589 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:01,589 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:01,591 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:01,592 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 0C959E3FFCA1491C9744AD171AD858D8
2016-01-14 17:40:01,592 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"], id: 944E19AF285E48E29F988F286002E0FE
2016-01-14 17:40:01,593 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSwap/swap"]
2016-01-14 17:40:01,593 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:01,593 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:01,594 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EB03C3AB040044CA8F3557691A844B1B, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:01,596 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:01,597 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:01,598 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793201307_0001_1_00_000000_0
2016-01-14 17:40:01,598 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793201307_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 17:40:01,599 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793201307_0001_1_00_000000_0
2016-01-14 17:40:01,599 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EB03C3AB040044CA8F3557691A844B1B, mem on close (mb), free: 171, total: 461, max: 672
2016-01-14 17:40:01 Completed running task attempt: attempt_1452793201307_0001_1_00_000000_0
2016-01-14 17:40:01,601 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793201307_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:01,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1452793201594, Process_Duration=5, Process_End_Time=1452793201599, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-01-14 17:40:01,602 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:01,602 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:01,604 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:01,603 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EB03C3AB040044CA8F3557691A844B1B, taskAttemptId=attempt_1452793201307_0001_1_00_000000_0, startTime=1452793201553, finishTime=1452793201602, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793201594, Process_Duration=5, Process_End_Time=1452793201599, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 17:40:01,604 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793201307_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:01,604 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:01,604 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793201307_0001_1_00_000000_0
2016-01-14 17:40:01,604 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793201307_0001_00_000001
2016-01-14 17:40:01,605 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:01,605 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:TASK_FINISHED]: vertexName=EB03C3AB040044CA8F3557691A844B1B, taskId=task_1452793201307_0001_1_00_000000, startTime=1452793201553, finishTime=1452793201605, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1452793201307_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793201594, Process_Duration=5, Process_End_Time=1452793201599, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 17:40:01,606 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793201307_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:01,606 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B] : 1
2016-01-14 17:40:01,606 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:VERTEX_FINISHED]: vertexName=EB03C3AB040044CA8F3557691A844B1B, vertexId=vertex_1452793201307_0001_1_00, initRequestedTime=1452793201510, initedTime=1452793201535, startRequestedTime=1452793201538, startedTime=1452793201538, finishTime=1452793201606, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793201594, Process_Duration=5, Process_End_Time=1452793201599, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1452793201553, firstTasksToStart=[ task_1452793201307_0001_1_00_000000 ], lastTaskFinishTime=1452793201602, lastTasksToFinish=[ task_1452793201307_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793201307_0001_1_00_000000 ], longestDurationTasks=[ task_1452793201307_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:01,608 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:01,608 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201307_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:01,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:01,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793201307_0001_00_000001, stoppedTime=1452793201609, exitStatus=0
2016-01-14 17:40:01,609 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201307_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:01,609 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 17:40:01,609 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793201307_0001_1
2016-01-14 17:40:01,610 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793201307_0001_1
2016-01-14 17:40:01,610 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 944E19AF285E48E29F988F286002E0FE for vertex: vertex_1452793201307_0001_1_00 [EB03C3AB040044CA8F3557691A844B1B]
2016-01-14 17:40:01,610 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:01,616 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793201307_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:01,616 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793201307_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:01,616 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793201307_0001_00_000001, Exiting
2016-01-14 17:40:01,616 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793201307_0001_00_000001
2016-01-14 17:40:01,617 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793201307_0001_00_000001
2016-01-14 17:40:01,619 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793201307_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:01,619 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793201307_0001_00_000001 completed successfully
2016-01-14 17:40:01,620 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793201307_0001_1, queueSize=0
2016-01-14 17:40:01,620 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:01,621 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201307_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793201307_0001_1, startTime=1452793201509, finishTime=1452793201618, timeTaken=109, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793201594, Process_Duration=5, Process_End_Time=1452793201599, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 17:40:01,621 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793201307_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:01,621 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793201307_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:01,621 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793201307_0001_00_000001
2016-01-14 17:40:01,621 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201307_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:01 Completed Dag: dag_1452793201307_0001_1
2016-01-14 17:40:01,622 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793201307_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:01,623 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793201307_0001_1_post
2016-01-14 17:40:01,623 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:01,623 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:01,630 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...pesplatform/testSwap/swap, applicationId=application_1452793201307_0001
2016-01-14 17:40:01,635 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:01,635 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:01,635 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:01,635 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:01,635 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:01,636 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:01,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:01,637 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:01,637 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:01,637 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:01,638 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:01,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:01,639 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:01,646 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/9D74FC0D2800488DA3AD7B232D9A8910/EDC1C7CAD6824010B243E3EB8C54BCB0/.tez/application_1452793201307_0001
2016-01-14 17:40:01,647 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.363
2016-01-14 17:40:01,691 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSwap/swap
2016-01-14 17:40:01,722 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:01,722 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:01,774 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-01-14 17:40:01,777 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 17:40:01,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 17:40:01,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 17:40:01,779 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:01,797 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower+upper.txt
2016-01-14 17:40:01,798 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-01-14 17:40:01,816 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 17:40:01,816 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-01-14 17:40:01,816 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-01-14 17:40:01,816 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 17:40:01,817 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 17:40:01,817 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 17:40:01,817 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...orm/testUnGroup/ungrouped
2016-01-14 17:40:01,818 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:01,836 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 17:40:01,841 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:01,841 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:01,841 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:01,852 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE/.tez/application_1452793201841_0001 doesn't exist and is created
2016-01-14 17:40:01,866 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:01,866 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:01,867 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE/.tez/application_1452793201841_0001
2016-01-14 17:40:01,870 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793201841_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:01,884 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:01,884 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:01,887 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:01,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:01,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE/.tez/application_1452793201841_0001/localmode-log-dir
2016-01-14 17:40:01,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE/.tez/application_1452793201841_0001/localmode-log-dir/history.txt.appattempt_1452793201841_0001_000000, maxErrors=10
2016-01-14 17:40:01,888 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:01,889 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793201841_0001_000000, appSubmitTime=1452793201869, launchTime=1452793201870
2016-01-14 17:40:01,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:01,898 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:39614
2016-01-14 17:40:01,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:01,900 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:01,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:01,904 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:01,906 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793201841_0001_000000, startTime=1452793201906
2016-01-14 17:40:01,906 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:01,906 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:01,967 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:01,967 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:01,967 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1452793201841_0001, dagName=[52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 17:40:01,987 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 17:40:01,990 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793201841_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE/.tez/application_1452793201841_0001/localmode-log-dir/dag_1452793201841_0001_1.dot
2016-01-14 17:40:01,992 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE/.tez/application_1452793201841_0001/localmode-log-dir/dag_1452793201841_0001_1-tez-dag.pb.txt
2016-01-14 17:40:02,018 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793201841_0001_1
2016-01-14 17:40:02,018 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:02,018 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 17:40:02 Running Dag: dag_1452793201841_0001_1
2016-01-14 17:40:02,028 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793201841_0001_1, submitTime=1452793201988
2016-01-14 17:40:02,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:02,029 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FAE8043452B6488EAAE270EB47C8057E
2016-01-14 17:40:02,029 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:02,030 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:02,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793201841_0001_1, initTime=1452793202028
2016-01-14 17:40:02,030 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793201841_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:02,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:DAG_STARTED]: dagID=dag_1452793201841_0001_1, startTime=1452793202031
2016-01-14 17:40:02,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:02,031 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:02,031 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793201841_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:02,032 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FAE8043452B6488EAAE270EB47C8057E : {C78DB9CF42D547CAA90801677B85CB9E={InputName=C78DB9CF42D547CAA90801677B85CB9E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:02,032 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C78DB9CF42D547CAA90801677B85CB9E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:02,032 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,032 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,032 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,033 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]. Starting root input initializers: 1
2016-01-14 17:40:02,033 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1452793201841_0001, dagName=[52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE] test/(1/1) ...orm/testUnGroup/ungrouped
2016-01-14 17:40:02,033 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:02,033 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C78DB9CF42D547CAA90801677B85CB9E on vertex vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,036 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:02,036 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C78DB9CF42D547CAA90801677B85CB9E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:02,038 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:02,038 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:02,041 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 412
2016-01-14 17:40:02,043 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C78DB9CF42D547CAA90801677B85CB9E on vertex vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,043 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:02,044 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C78DB9CF42D547CAA90801677B85CB9E=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:02,044 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E] parallelism set to 1
2016-01-14 17:40:02,044 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:02,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793201841_0001_1_00, eventCount=1
2016-01-14 17:40:02,044 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,044 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D311D951C44F4BBEBF2CF648D6ECE2EA, vertexId=vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:02,044 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D311D951C44F4BBEBF2CF648D6ECE2EA, vertexId=vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,046 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D311D951C44F4BBEBF2CF648D6ECE2EA, vertexName=FAE8043452B6488EAAE270EB47C8057E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:02,046 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D311D951C44F4BBEBF2CF648D6ECE2EA, vertexId=vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:02,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FAE8043452B6488EAAE270EB47C8057E, vertexId=vertex_1452793201841_0001_1_00, initRequestedTime=1452793202032, initedTime=1452793202047, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:02,048 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E] with distanceFromRoot: 0
2016-01-14 17:40:02,048 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:02,048 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FAE8043452B6488EAAE270EB47C8057E
2016-01-14 17:40:02,049 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:02,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793201841_0001_1_00, startRequestedTime=1452793202048, startedTime=1452793202048
2016-01-14 17:40:02,049 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:02,050 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:02,050 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:TASK_STARTED]: vertexName=FAE8043452B6488EAAE270EB47C8057E, taskId=task_1452793201841_0001_1_00_000000, scheduledTime=1452793202050, launchTime=1452793202050
2016-01-14 17:40:02,050 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793201841_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:02,051 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,051 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: FAE8043452B6488EAAE270EB47C8057E, VertexParallelism: 1, TaskAttemptID:attempt_1452793201841_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C78DB9CF42D547CAA90801677B85CB9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D311D951C44F4BBEBF2CF648D6ECE2EA, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:02,051 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793201841_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:02,051 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:02,052 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:02,053 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:02,053 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:02,054 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793201841_0001_00_000001
2016-01-14 17:40:02,054 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201841_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:02,056 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793201841_0001_00_000001
2016-01-14 17:40:02,056 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:02,057 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201841_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:02,058 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:02,058 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793201841_0001_00_000001, launchTime=1452793202058
2016-01-14 17:40:02,059 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793201841_0001_00_000001
2016-01-14 17:40:02,059 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793201841_0001_1_00_000000_0] to container: [container_1452793201841_0001_00_000001]
2016-01-14 17:40:02,060 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201841_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:02,060 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793201841_0001_00_000001 given task: attempt_1452793201841_0001_1_00_000000_0
2016-01-14 17:40:02,060 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793201841_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793201841_0001_1_00_000000_0
2016-01-14 17:40:02,060 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793201841_0001_1_00_000000_0] started. Is using containerId: [container_1452793201841_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:02,060 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:02,060 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FAE8043452B6488EAAE270EB47C8057E, taskAttemptId=attempt_1452793201841_0001_1_00_000000_0, startTime=1452793202060, containerId=container_1452793201841_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793201841_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:02,061 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793201841_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:02,061 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793201841_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:02 Starting to run new task attempt: attempt_1452793201841_0001_1_00_000000_0
2016-01-14 17:40:02,062 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793201841_0001_1_00_000000_0
2016-01-14 17:40:02,062 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:02,062 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:02,062 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:02,062 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:02,062 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:02,063 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:02,063 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE] test/(1/1) ...orm/testUnGroup/ungrouped, VertexName: FAE8043452B6488EAAE270EB47C8057E, VertexParallelism: 1, TaskAttemptID:attempt_1452793201841_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C78DB9CF42D547CAA90801677B85CB9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D311D951C44F4BBEBF2CF648D6ECE2EA, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:02,063 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:02,064 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793201841_0001_1_00_000000_0
2016-01-14 17:40:02,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:02,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:02,065 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793201841_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:02,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C78DB9CF42D547CAA90801677B85CB9E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:02,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:02,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C78DB9CF42D547CAA90801677B85CB9E
2016-01-14 17:40:02,067 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C78DB9CF42D547CAA90801677B85CB9E
2016-01-14 17:40:02,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:02,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D311D951C44F4BBEBF2CF648D6ECE2EA, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:02,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:02,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D311D951C44F4BBEBF2CF648D6ECE2EA
2016-01-14 17:40:02,069 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:02,069 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D311D951C44F4BBEBF2CF648D6ECE2EA
2016-01-14 17:40:02,071 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:02,073 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452793201841_0001, with dag id: dag_1452793201841_0001_1
2016-01-14 17:40:02,079 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:02,080 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C78DB9CF42D547CAA90801677B85CB9E
2016-01-14 17:40:02,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C78DB9CF42D547CAA90801677B85CB9E
2016-01-14 17:40:02,084 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:02,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D311D951C44F4BBEBF2CF648D6ECE2EA
2016-01-14 17:40:02,088 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FAE8043452B6488EAAE270EB47C8057E, ordinal: 0
2016-01-14 17:40:02,088 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FAE8043452B6488EAAE270EB47C8057E, mem on start (mb), free: 86, total: 461, max: 672
2016-01-14 17:40:02,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:02,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:02,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:02,089 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:02,089 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:02,089 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:02,089 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:02,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:02,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:02,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:02,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 17:40:02,090 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C78DB9CF42D547CAA90801677B85CB9E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:02,091 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D311D951C44F4BBEBF2CF648D6ECE2EA, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:02,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:02,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C78DB9CF42D547CAA90801677B85CB9E
2016-01-14 17:40:02,091 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:02,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C78DB9CF42D547CAA90801677B85CB9E
2016-01-14 17:40:02,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C78DB9CF42D547CAA90801677B85CB9E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:02,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:02,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:02,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:02,093 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793201841_0001_1_00_000000_0
2016-01-14 17:40:02,093 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:02,095 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@24fc01e3
2016-01-14 17:40:02,095 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-01-14 17:40:02,096 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"] streamed: true, id: C78DB9CF42D547CAA90801677B85CB9E
2016-01-14 17:40:02,096 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"], id: D311D951C44F4BBEBF2CF648D6ECE2EA
2016-01-14 17:40:02,096 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@24fc01e3
2016-01-14 17:40:02,097 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:02,097 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:02,097 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped"]
2016-01-14 17:40:02,098 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower+upper.txt"]
2016-01-14 17:40:02,098 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:02,099 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FAE8043452B6488EAAE270EB47C8057E, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:02,102 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:02,102 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:02,103 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793201841_0001_1_00_000000_0
2016-01-14 17:40:02,103 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793201841_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 17:40:02,104 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793201841_0001_1_00_000000_0
2016-01-14 17:40:02,105 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FAE8043452B6488EAAE270EB47C8057E, mem on close (mb), free: 85, total: 461, max: 672
2016-01-14 17:40:02 Completed running task attempt: attempt_1452793201841_0001_1_00_000000_0
2016-01-14 17:40:02,106 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793201841_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:02,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:02,109 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FAE8043452B6488EAAE270EB47C8057E, taskAttemptId=attempt_1452793201841_0001_1_00_000000_0, startTime=1452793202060, finishTime=1452793202107, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793202099, Process_Duration=5, Process_End_Time=1452793202104, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-01-14 17:40:02,109 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793201841_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:02,109 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793201841_0001_1_00_000000_0
2016-01-14 17:40:02,109 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:02,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1452793202099, Process_Duration=5, Process_End_Time=1452793202104, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5, Tuples_Written=10]]
2016-01-14 17:40:02,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:TASK_FINISHED]: vertexName=FAE8043452B6488EAAE270EB47C8057E, taskId=task_1452793201841_0001_1_00_000000, startTime=1452793202060, finishTime=1452793202109, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452793201841_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793202099, Process_Duration=5, Process_End_Time=1452793202104, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-01-14 17:40:02,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:02,111 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:02,111 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:02,110 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793201841_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:02,111 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793201841_0001_00_000001
2016-01-14 17:40:02,112 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E] : 1
2016-01-14 17:40:02,112 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,114 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:02,114 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:VERTEX_FINISHED]: vertexName=FAE8043452B6488EAAE270EB47C8057E, vertexId=vertex_1452793201841_0001_1_00, initRequestedTime=1452793202032, initedTime=1452793202047, startRequestedTime=1452793202048, startedTime=1452793202048, finishTime=1452793202112, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793202099, Process_Duration=5, Process_End_Time=1452793202104, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10, vertexStats=firstTaskStartTime=1452793202060, firstTasksToStart=[ task_1452793201841_0001_1_00_000000 ], lastTaskFinishTime=1452793202107, lastTasksToFinish=[ task_1452793201841_0001_1_00_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793201841_0001_1_00_000000 ], longestDurationTasks=[ task_1452793201841_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:02,114 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:02,115 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201841_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:02,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793201841_0001_00_000001, stoppedTime=1452793202115, exitStatus=0
2016-01-14 17:40:02,115 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201841_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:02,116 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:02,116 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 17:40:02,116 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793201841_0001_1
2016-01-14 17:40:02,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793201841_0001_1
2016-01-14 17:40:02,117 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D311D951C44F4BBEBF2CF648D6ECE2EA for vertex: vertex_1452793201841_0001_1_00 [FAE8043452B6488EAAE270EB47C8057E]
2016-01-14 17:40:02,122 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793201841_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:02,122 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793201841_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:02,122 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793201841_0001_00_000001, Exiting
2016-01-14 17:40:02,122 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793201841_0001_00_000001
2016-01-14 17:40:02,122 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793201841_0001_00_000001
2016-01-14 17:40:02,124 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793201841_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:02,125 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793201841_0001_00_000001 completed successfully
2016-01-14 17:40:02,126 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793201841_0001_1, queueSize=0
2016-01-14 17:40:02,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:02,127 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793201841_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793201841_0001_1, startTime=1452793202031, finishTime=1452793202124, timeTaken=93, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=45, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=483393536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793202099, Process_Duration=5, Process_End_Time=1452793202104, Read_Duration=2, Tuples_Read=5, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=10
2016-01-14 17:40:02,127 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793201841_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:02,127 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793201841_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:02,128 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793201841_0001_00_000001
2016-01-14 17:40:02,128 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793201841_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:02 Completed Dag: dag_1452793201841_0001_1
2016-01-14 17:40:02,128 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793201841_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:02,128 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793201841_0001_1_post
2016-01-14 17:40:02,128 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:02,129 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:02,137 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...orm/testUnGroup/ungrouped, applicationId=application_1452793201841_0001
2016-01-14 17:40:02,141 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:02,141 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:02,141 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:02,141 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:02,141 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:02,141 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:02,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:02,143 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:02,143 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:02,143 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:02,143 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:02,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:02,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:02,149 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/52C822DAA5EF411795688A3ADACA2435/DD0A5C55B29B49CBB241C15E5F06A0EE/.tez/application_1452793201841_0001
2016-01-14 17:40:02,149 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.333
2016-01-14 17:40:02,188 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testUnGroup/ungrouped
2016-01-14 17:40:02,231 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:02,232 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:02,297 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-01-14 17:40:02,317 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.050
2016-01-14 17:40:02,318 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 17:40:02,318 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 17:40:02,318 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:02,340 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-01-14 17:40:02,356 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 17:40:02,365 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-01-14 17:40:02,469 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting
2016-01-14 17:40:02,469 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 17:40:02,469 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:02,470 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-01-14 17:40:02,470 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] parallel execution of steps is enabled: false
2016-01-14 17:40:02,470 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executing total steps: 1
2016-01-14 17:40:02,470 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] allocating management threads: 1
2016-01-14 17:40:02,471 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting step: (1/1) ...lexcogrouprelativeunknown
2016-01-14 17:40:02,471 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:02,485 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tez session mode enabled: true
2016-01-14 17:40:02,490 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:02,491 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:02,491 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:02,500 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10/.tez/application_1452793202490_0001 doesn't exist and is created
2016-01-14 17:40:02,513 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:02,514 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:02,514 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10/.tez/application_1452793202490_0001
2016-01-14 17:40:02,516 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793202490_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:02,529 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:02,530 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:02,532 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:02,532 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:02,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10/.tez/application_1452793202490_0001/localmode-log-dir
2016-01-14 17:40:02,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10/.tez/application_1452793202490_0001/localmode-log-dir/history.txt.appattempt_1452793202490_0001_000000, maxErrors=10
2016-01-14 17:40:02,533 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:02,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793202490_0001_000000, appSubmitTime=1452793202516, launchTime=1452793202516
2016-01-14 17:40:02,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:02,540 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:02,543 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:41083
2016-01-14 17:40:02,545 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:02,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:02,547 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:02,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793202490_0001_000000, startTime=1452793202549
2016-01-14 17:40:02,549 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:02,549 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:02,614 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:02,614 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:02,614 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1452793202490_0001, dagName=[3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 17:40:02,635 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 17:40:02,638 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793202490_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10/.tez/application_1452793202490_0001/localmode-log-dir/dag_1452793202490_0001_1.dot
2016-01-14 17:40:02,640 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10/.tez/application_1452793202490_0001/localmode-log-dir/dag_1452793202490_0001_1-tez-dag.pb.txt
2016-01-14 17:40:02,674 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793202490_0001_1
2016-01-14 17:40:02,674 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:02,674 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 17:40:02 Running Dag: dag_1452793202490_0001_1
2016-01-14 17:40:02,685 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793202490_0001_1, submitTime=1452793202635
2016-01-14 17:40:02,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:02,687 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EA52DC3DE5634348B1A9EF657EF385CF
2016-01-14 17:40:02,687 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:02,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793202490_0001_1, initTime=1452793202686
2016-01-14 17:40:02,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:02,690 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793202490_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:02,691 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:02,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:DAG_STARTED]: dagID=dag_1452793202490_0001_1, startTime=1452793202691
2016-01-14 17:40:02,691 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:02,692 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793202490_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:02,692 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AA2CEC34FD774094B4CE54270A5D62EC : {EE844AC8D5184015A7D01B5B5909F915={InputName=EE844AC8D5184015A7D01B5B5909F915}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:02,692 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: EE844AC8D5184015A7D01B5B5909F915, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:02,692 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC]
2016-01-14 17:40:02,692 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC]
2016-01-14 17:40:02,692 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1452793202490_0001, dagName=[3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10] lower*upper/(1/1) ...lexcogrouprelativeunknown
2016-01-14 17:40:02,692 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC]
2016-01-14 17:40:02,693 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC]. Starting root input initializers: 1
2016-01-14 17:40:02,694 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:02,694 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E8968325287246B492DEEF5D5F88A88C : {E5A288B319BF466999C0368EC14C0D39={InputName=E5A288B319BF466999C0368EC14C0D39}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:02,694 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E5A288B319BF466999C0368EC14C0D39, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:02,694 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C]
2016-01-14 17:40:02,694 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C]
2016-01-14 17:40:02,694 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C]
2016-01-14 17:40:02,695 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C]. Starting root input initializers: 1
2016-01-14 17:40:02,695 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: EE844AC8D5184015A7D01B5B5909F915 on vertex vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC]
2016-01-14 17:40:02,700 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:02,701 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF]
2016-01-14 17:40:02,703 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:02,703 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input EE844AC8D5184015A7D01B5B5909F915 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:02,705 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:02,705 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:02,707 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:40:02,708 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF]
2016-01-14 17:40:02,708 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF]
2016-01-14 17:40:02,708 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF]
2016-01-14 17:40:02,708 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=603769BFD376413B9E6575B09827A108, vertexId=vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:02,708 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=603769BFD376413B9E6575B09827A108, vertexId=vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF]
2016-01-14 17:40:02,712 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 17:40:02,712 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=603769BFD376413B9E6575B09827A108, vertexName=EA52DC3DE5634348B1A9EF657EF385CF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:02,712 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=603769BFD376413B9E6575B09827A108, vertexId=vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF]
2016-01-14 17:40:02,714 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: EE844AC8D5184015A7D01B5B5909F915 on vertex vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC]
2016-01-14 17:40:02,714 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E5A288B319BF466999C0368EC14C0D39 on vertex vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C]
2016-01-14 17:40:02,715 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EA52DC3DE5634348B1A9EF657EF385CF, vertexId=vertex_1452793202490_0001_1_02, initRequestedTime=1452793202701, initedTime=1452793202715, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:40:02,715 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:02,716 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:40:02,717 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:02,717 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {EE844AC8D5184015A7D01B5B5909F915=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:02,717 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC] parallelism set to 1
2016-01-14 17:40:02,718 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:02,718 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793202490_0001_1_00, eventCount=1
2016-01-14 17:40:02,722 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:02,722 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AA2CEC34FD774094B4CE54270A5D62EC, vertexId=vertex_1452793202490_0001_1_00, initRequestedTime=1452793202692, initedTime=1452793202719, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:02,722 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC] with distanceFromRoot: 0
2016-01-14 17:40:02,722 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:02,723 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AA2CEC34FD774094B4CE54270A5D62EC
2016-01-14 17:40:02,723 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC]
2016-01-14 17:40:02,724 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:02,725 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793202490_0001_1_00, startRequestedTime=1452793202723, startedTime=1452793202723
2016-01-14 17:40:02,725 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:02,725 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:02,726 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_STARTED]: vertexName=AA2CEC34FD774094B4CE54270A5D62EC, taskId=task_1452793202490_0001_1_00_000000, scheduledTime=1452793202725, launchTime=1452793202725
2016-01-14 17:40:02,726 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793202490_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:02,726 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793202490_0001_1_00 for vertex: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF] numStartedSources: 1 numSources: 2
2016-01-14 17:40:02,726 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF] numStartedSources: 1 numSources: 2
2016-01-14 17:40:02,726 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC]
2016-01-14 17:40:02,727 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: AA2CEC34FD774094B4CE54270A5D62EC, VertexParallelism: 1, TaskAttemptID:attempt_1452793202490_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EE844AC8D5184015A7D01B5B5909F915, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EA52DC3DE5634348B1A9EF657EF385CF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:02,727 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:02,728 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E5A288B319BF466999C0368EC14C0D39 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:02,728 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:02,729 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:02,729 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793202490_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:02,729 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:02,731 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 17:40:02,731 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] submitted tez dag to app master: application_1452793202490_0001, with dag id: dag_1452793202490_0001_1
2016-01-14 17:40:02,733 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E5A288B319BF466999C0368EC14C0D39 on vertex vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C]
2016-01-14 17:40:02,734 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:02,735 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E5A288B319BF466999C0368EC14C0D39=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:02,735 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C] parallelism set to 1
2016-01-14 17:40:02,735 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:02,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793202490_0001_1_01, eventCount=1
2016-01-14 17:40:02,736 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:02,736 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E8968325287246B492DEEF5D5F88A88C, vertexId=vertex_1452793202490_0001_1_01, initRequestedTime=1452793202694, initedTime=1452793202736, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:02,736 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C] with distanceFromRoot: 0
2016-01-14 17:40:02,736 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:02,737 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E8968325287246B492DEEF5D5F88A88C
2016-01-14 17:40:02,737 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C]
2016-01-14 17:40:02,734 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:02,737 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793202490_0001_1_01, startRequestedTime=1452793202737, startedTime=1452793202737
2016-01-14 17:40:02,737 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:02,738 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:02,739 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:02,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_STARTED]: vertexName=E8968325287246B492DEEF5D5F88A88C, taskId=task_1452793202490_0001_1_01_000000, scheduledTime=1452793202739, launchTime=1452793202739
2016-01-14 17:40:02,740 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793202490_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:02,741 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:02,742 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:02,743 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793202490_0001_00_000001
2016-01-14 17:40:02,744 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:02,744 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793202490_0001_1_01 for vertex: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF] numStartedSources: 2 numSources: 2
2016-01-14 17:40:02,745 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF] with distanceFromRoot: 1
2016-01-14 17:40:02,745 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C]
2016-01-14 17:40:02,749 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793202490_0001_00_000001
2016-01-14 17:40:02,750 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:02,750 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E8968325287246B492DEEF5D5F88A88C in vertex: EA52DC3DE5634348B1A9EF657EF385CF
2016-01-14 17:40:02,750 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AA2CEC34FD774094B4CE54270A5D62EC in vertex: EA52DC3DE5634348B1A9EF657EF385CF
2016-01-14 17:40:02,752 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EA52DC3DE5634348B1A9EF657EF385CF with 2 source tasks and 1 pending tasks
2016-01-14 17:40:02,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793202490_0001_1_02, startRequestedTime=1452793202750, startedTime=1452793202750
2016-01-14 17:40:02,752 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:02,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793202490_0001_00_000001, launchTime=1452793202751
2016-01-14 17:40:02,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:02,755 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: E8968325287246B492DEEF5D5F88A88C, VertexParallelism: 1, TaskAttemptID:attempt_1452793202490_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E5A288B319BF466999C0368EC14C0D39, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EA52DC3DE5634348B1A9EF657EF385CF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:02,755 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:02,755 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793202490_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:02,755 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:02,756 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793202490_0001_00_000001
2016-01-14 17:40:02,757 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:02,758 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793202490_0001_1_00_000000_0] to container: [container_1452793202490_0001_00_000001]
2016-01-14 17:40:02,758 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:02,758 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793202490_0001_00_000001 given task: attempt_1452793202490_0001_1_00_000000_0
2016-01-14 17:40:02,758 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793202490_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793202490_0001_1_00_000000_0
2016-01-14 17:40:02,759 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793202490_0001_00_000002
2016-01-14 17:40:02,759 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:02,760 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793202490_0001_1_00_000000_0] started. Is using containerId: [container_1452793202490_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:02 Starting to run new task attempt: attempt_1452793202490_0001_1_00_000000_0
2016-01-14 17:40:02,760 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793202490_0001_1_00_000000_0
2016-01-14 17:40:02,760 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:02,761 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:02,761 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:02,761 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:02,761 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:02,761 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:02,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: AA2CEC34FD774094B4CE54270A5D62EC, VertexParallelism: 1, TaskAttemptID:attempt_1452793202490_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EE844AC8D5184015A7D01B5B5909F915, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EA52DC3DE5634348B1A9EF657EF385CF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:02,762 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:02,762 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:02,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AA2CEC34FD774094B4CE54270A5D62EC, taskAttemptId=attempt_1452793202490_0001_1_00_000000_0, startTime=1452793202759, containerId=container_1452793202490_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793202490_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:02,762 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793202490_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:02,764 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793202490_0001_1_00_000000_0
2016-01-14 17:40:02,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:02,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:02,765 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793202490_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:02,767 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793202490_0001_00_000002
2016-01-14 17:40:02,767 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:02,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EE844AC8D5184015A7D01B5B5909F915, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:02,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:02,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EE844AC8D5184015A7D01B5B5909F915
2016-01-14 17:40:02,771 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793202490_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:02,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:02,773 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:02,774 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793202490_0001_00_000002, launchTime=1452793202772
2016-01-14 17:40:02,774 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:02,777 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: EE844AC8D5184015A7D01B5B5909F915
2016-01-14 17:40:02,778 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793202490_0001_00_000002
2016-01-14 17:40:02,780 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793202490_0001_1_01_000000_0] to container: [container_1452793202490_0001_00_000002]
2016-01-14 17:40:02,781 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:02,781 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:02,782 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793202490_0001_00_000002 given task: attempt_1452793202490_0001_1_01_000000_0
2016-01-14 17:40:02,783 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793202490_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793202490_0001_1_01_000000_0
2016-01-14 17:40:02,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EA52DC3DE5634348B1A9EF657EF385CF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:02,782 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793202490_0001_1_01_000000_0] started. Is using containerId: [container_1452793202490_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:40:02,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E8968325287246B492DEEF5D5F88A88C, taskAttemptId=attempt_1452793202490_0001_1_01_000000_0, startTime=1452793202781, containerId=container_1452793202490_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793202490_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:40:02,786 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:02,786 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: EE844AC8D5184015A7D01B5B5909F915
2016-01-14 17:40:02,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EE844AC8D5184015A7D01B5B5909F915
2016-01-14 17:40:02,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:02,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EA52DC3DE5634348B1A9EF657EF385CF
2016-01-14 17:40:02 Starting to run new task attempt: attempt_1452793202490_0001_1_01_000000_0
2016-01-14 17:40:02,790 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793202490_0001_1_01_000000_0
2016-01-14 17:40:02,790 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:02,790 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:02,790 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:02,790 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:02,791 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:02,791 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:02,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: E8968325287246B492DEEF5D5F88A88C, VertexParallelism: 1, TaskAttemptID:attempt_1452793202490_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E5A288B319BF466999C0368EC14C0D39, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EA52DC3DE5634348B1A9EF657EF385CF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:02,791 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:02,793 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AA2CEC34FD774094B4CE54270A5D62EC, ordinal: 0
2016-01-14 17:40:02,793 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AA2CEC34FD774094B4CE54270A5D62EC, mem on start (mb), free: 174, total: 442, max: 672
2016-01-14 17:40:02,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:02,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:02,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:02,794 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:02,794 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793202490_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:02,795 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793202490_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:02,796 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793202490_0001_1_01_000000_0
2016-01-14 17:40:02,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:02,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:02,800 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:02,800 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793202490_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:40:02,802 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:02,802 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EA52DC3DE5634348B1A9EF657EF385CF
2016-01-14 17:40:02,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EA52DC3DE5634348B1A9EF657EF385CF
2016-01-14 17:40:02,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E5A288B319BF466999C0368EC14C0D39, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:02,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:02,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E5A288B319BF466999C0368EC14C0D39
2016-01-14 17:40:02,806 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E5A288B319BF466999C0368EC14C0D39
2016-01-14 17:40:02,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EA52DC3DE5634348B1A9EF657EF385CF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:02,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:02,806 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:02,811 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:02,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:02,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EA52DC3DE5634348B1A9EF657EF385CF
2016-01-14 17:40:02,811 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:02,812 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:02,812 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:02,812 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:02,812 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:02,812 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EE844AC8D5184015A7D01B5B5909F915, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:02,812 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EA52DC3DE5634348B1A9EF657EF385CF, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:02,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:02,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EE844AC8D5184015A7D01B5B5909F915 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:02,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:02,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:02,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EE844AC8D5184015A7D01B5B5909F915
2016-01-14 17:40:02,813 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:02,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EE844AC8D5184015A7D01B5B5909F915
2016-01-14 17:40:02,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:02,814 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793202490_0001_1_00_000000_0
2016-01-14 17:40:02,818 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:02,818 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EA52DC3DE5634348B1A9EF657EF385CF
2016-01-14 17:40:02,823 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E8968325287246B492DEEF5D5F88A88C, ordinal: 1
2016-01-14 17:40:02,824 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8968325287246B492DEEF5D5F88A88C, mem on start (mb), free: 170, total: 442, max: 672
2016-01-14 17:40:02,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:02,825 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:02,823 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:02,826 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5c0f8abd
2016-01-14 17:40:02,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EA52DC3DE5634348B1A9EF657EF385CF
2016-01-14 17:40:02,828 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 17:40:02,829 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5c0f8abd
2016-01-14 17:40:02,829 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:02,829 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:02,829 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:02,830 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E5A288B319BF466999C0368EC14C0D39
2016-01-14 17:40:02,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E5A288B319BF466999C0368EC14C0D39
2016-01-14 17:40:02,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:02,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:02,831 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:02,832 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:02,832 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:02,833 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:02,833 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:02,834 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:02,834 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E5A288B319BF466999C0368EC14C0D39, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:02,834 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EA52DC3DE5634348B1A9EF657EF385CF, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:02,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:02,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E5A288B319BF466999C0368EC14C0D39 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:02,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:02,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:02,835 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: EE844AC8D5184015A7D01B5B5909F915
2016-01-14 17:40:02,837 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: D2977B3A924740AB91DA576B49682034
2016-01-14 17:40:02,837 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] D2977B3A924740AB91DA576B49682034
2016-01-14 17:40:02,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E5A288B319BF466999C0368EC14C0D39
2016-01-14 17:40:02,838 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:02,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E5A288B319BF466999C0368EC14C0D39
2016-01-14 17:40:02,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:02,840 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@21c1f85c; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@45987154; comparator=cascading.tuple.hadoop.util.TupleComparator@1b38657e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:02,841 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:02,841 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:02,842 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:02,846 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793202490_0001_1_01_000000_0
2016-01-14 17:40:02,853 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:02,856 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@377c4d48
2016-01-14 17:40:02,858 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:02,859 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:02,859 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:02,860 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:02,861 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:02,861 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@377c4d48
2016-01-14 17:40:02,862 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:02,862 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:02,866 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:02,867 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: E5A288B319BF466999C0368EC14C0D39
2016-01-14 17:40:02,868 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]], id: D2977B3A924740AB91DA576B49682034
2016-01-14 17:40:02,868 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] D2977B3A924740AB91DA576B49682034
2016-01-14 17:40:02,869 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@26ac9bba; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@e23e42c; comparator=cascading.tuple.hadoop.util.TupleComparator@56590bf4; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:02,869 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 17:40:02,870 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:02,871 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AA2CEC34FD774094B4CE54270A5D62EC, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:02,871 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:02,871 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:02,872 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:03,287 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793202490_0001_1_00_000000_0
2016-01-14 17:40:03,287 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AA2CEC34FD774094B4CE54270A5D62EC, mem on close (mb), free: 330, total: 550, max: 672
2016-01-14 17:40:03,287 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:03,287 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:03,288 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 17:40:03,288 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 17:40:03,290 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:03,299 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:03,299 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:03,299 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:03,300 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:03,300 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:03,301 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:03,302 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:03,302 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E8968325287246B492DEEF5D5F88A88C, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:03 Completed running task attempt: attempt_1452793202490_0001_1_00_000000_0
2016-01-14 17:40:03,303 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793202490_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:03,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793202490_0001_1_00, eventCount=1
2016-01-14 17:40:03,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=81, FILE_BYTES_WRITTEN=647, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=404, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793202871, Process_Duration=416, Process_End_Time=1452793203287, Read_Duration=414, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 17:40:03,306 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AA2CEC34FD774094B4CE54270A5D62EC, taskAttemptId=attempt_1452793202490_0001_1_00_000000_0, startTime=1452793202759, finishTime=1452793203305, timeTaken=546, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=81, FILE_BYTES_WRITTEN=647, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=404, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793202871, Process_Duration=416, Process_End_Time=1452793203287, Read_Duration=414, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:03,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:03,306 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793202490_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:03,306 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793202490_0001_1_00_000000_0
2016-01-14 17:40:03,307 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:03,307 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793202490_0001_1_01_000000_0
2016-01-14 17:40:03,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:03,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_FINISHED]: vertexName=AA2CEC34FD774094B4CE54270A5D62EC, taskId=task_1452793202490_0001_1_00_000000, startTime=1452793202759, finishTime=1452793203306, timeTaken=547, status=SUCCEEDED, successfulAttemptID=attempt_1452793202490_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=81, FILE_BYTES_WRITTEN=647, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=404, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793202871, Process_Duration=416, Process_End_Time=1452793203287, Read_Duration=414, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:03,307 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8968325287246B492DEEF5D5F88A88C, mem on close (mb), free: 327, total: 550, max: 672
2016-01-14 17:40:03,307 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793202490_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:03,307 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:03,308 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC] : 1
2016-01-14 17:40:03,308 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:03,308 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 17:40:03,308 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC]
2016-01-14 17:40:03,309 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 17:40:03,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:03,312 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_FINISHED]: vertexName=AA2CEC34FD774094B4CE54270A5D62EC, vertexId=vertex_1452793202490_0001_1_00, initRequestedTime=1452793202692, initedTime=1452793202719, startRequestedTime=1452793202723, startedTime=1452793202723, finishTime=1452793203308, timeTaken=585, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=81, FILE_BYTES_WRITTEN=647, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=404, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793202871, Process_Duration=416, Process_End_Time=1452793203287, Read_Duration=414, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452793202759, firstTasksToStart=[ task_1452793202490_0001_1_00_000000 ], lastTaskFinishTime=1452793203305, lastTasksToFinish=[ task_1452793202490_0001_1_00_000000 ], minTaskDuration=546, maxTaskDuration=546, avgTaskDuration=546.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793202490_0001_1_00_000000 ], longestDurationTasks=[ task_1452793202490_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:03,312 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:03,313 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:03,313 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:03,313 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:03,314 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:03,314 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF] attempt: attempt_1452793202490_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:03,317 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:03,317 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793202490_0001_00_000001, stoppedTime=1452793203316, exitStatus=0
2016-01-14 17:40:03,317 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:03,317 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 17:40:03,317 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793202490_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:40:03,318 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:03,318 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793202490_0001_00_000001
2016-01-14 17:40:03,319 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793202490_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:03,319 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793202490_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:03,319 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793202490_0001_00_000001, Exiting
2016-01-14 17:40:03,319 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793202490_0001_00_000001
2016-01-14 17:40:03,319 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793202490_0001_00_000001
2016-01-14 17:40:03 Completed running task attempt: attempt_1452793202490_0001_1_01_000000_0
2016-01-14 17:40:03,323 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793202490_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:03,323 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793202490_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:03,324 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793202490_0001_00_000001 completed successfully
2016-01-14 17:40:03,324 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793202490_0001_1_01, eventCount=1
2016-01-14 17:40:03,325 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:03,325 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:03,326 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E8968325287246B492DEEF5D5F88A88C, taskAttemptId=attempt_1452793202490_0001_1_01_000000_0, startTime=1452793202781, finishTime=1452793203325, timeTaken=544, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=11270, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=404, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793203302, Process_Duration=5, Process_End_Time=1452793203307, Read_Duration=5, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:03,326 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793202490_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:03,326 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:03,326 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793202490_0001_1_01_000000_0
2016-01-14 17:40:03,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_FINISHED]: vertexName=E8968325287246B492DEEF5D5F88A88C, taskId=task_1452793202490_0001_1_01_000000, startTime=1452793202781, finishTime=1452793203328, timeTaken=547, status=SUCCEEDED, successfulAttemptID=attempt_1452793202490_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=11270, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=404, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793203302, Process_Duration=5, Process_End_Time=1452793203307, Read_Duration=5, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:03,329 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793202490_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:03,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=11270, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=404, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793203302, Process_Duration=5, Process_End_Time=1452793203307, Read_Duration=5, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 17:40:03,330 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C] : 1
2016-01-14 17:40:03,330 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C]
2016-01-14 17:40:03,330 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:03,330 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:03,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:03,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_FINISHED]: vertexName=E8968325287246B492DEEF5D5F88A88C, vertexId=vertex_1452793202490_0001_1_01, initRequestedTime=1452793202694, initedTime=1452793202736, startRequestedTime=1452793202737, startedTime=1452793202737, finishTime=1452793203330, timeTaken=593, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=11270, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=404, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793203302, Process_Duration=5, Process_End_Time=1452793203307, Read_Duration=5, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452793202781, firstTasksToStart=[ task_1452793202490_0001_1_01_000000 ], lastTaskFinishTime=1452793203325, lastTasksToFinish=[ task_1452793202490_0001_1_01_000000 ], minTaskDuration=544, maxTaskDuration=544, avgTaskDuration=544.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793202490_0001_1_01_000000 ], longestDurationTasks=[ task_1452793202490_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:03,332 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:03,332 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:03,332 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793202490_0001_00_000002
2016-01-14 17:40:03,333 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793202490_0001_00_000002, stoppedTime=1452793203332, exitStatus=0
2016-01-14 17:40:03,333 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:03,333 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF] attempt: attempt_1452793202490_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:03,333 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EA52DC3DE5634348B1A9EF657EF385CF
2016-01-14 17:40:03,333 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF]
2016-01-14 17:40:03,334 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 17:40:03,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:03,334 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793202490_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:40:03,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_STARTED]: vertexName=EA52DC3DE5634348B1A9EF657EF385CF, taskId=task_1452793202490_0001_1_02_000000, scheduledTime=1452793203334, launchTime=1452793203334
2016-01-14 17:40:03,334 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793202490_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:03,334 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF]
2016-01-14 17:40:03,335 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:03,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:03,335 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(221)) - No more tasks for container with id : container_1452793202490_0001_00_000002. Asking it to die
2016-01-14 17:40:03,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:03,335 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: EA52DC3DE5634348B1A9EF657EF385CF, VertexParallelism: 1, TaskAttemptID:attempt_1452793202490_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E8968325287246B492DEEF5D5F88A88C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=AA2CEC34FD774094B4CE54270A5D62EC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=603769BFD376413B9E6575B09827A108, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:03,335 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793202490_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:03,338 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793202490_0001_00_000002, Exiting
2016-01-14 17:40:03,338 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793202490_0001_00_000002
2016-01-14 17:40:03,338 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793202490_0001_00_000002
2016-01-14 17:40:03,338 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793202490_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:03,340 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793202490_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:03,340 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:03,340 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793202490_0001_00_000002 completed successfully
2016-01-14 17:40:03,340 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:03,341 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793202490_0001_00_000003
2016-01-14 17:40:03,341 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:03,345 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793202490_0001_00_000003
2016-01-14 17:40:03,347 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:40:03,353 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:03,353 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:03,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793202490_0001_00_000003, launchTime=1452793203353
2016-01-14 17:40:03,354 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793202490_0001_00_000003
2016-01-14 17:40:03,356 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793202490_0001_1_02_000000_0] to container: [container_1452793202490_0001_00_000003]
2016-01-14 17:40:03,356 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:03,357 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793202490_0001_1_02_000000_0] started. Is using containerId: [container_1452793202490_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 17:40:03,357 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EA52DC3DE5634348B1A9EF657EF385CF, taskAttemptId=attempt_1452793202490_0001_1_02_000000_0, startTime=1452793203357, containerId=container_1452793202490_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793202490_0001_00_000003/teamcity, completedLogs=
2016-01-14 17:40:03,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:03,358 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793202490_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:03,358 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793202490_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:03,360 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793202490_0001_00_000003 given task: attempt_1452793202490_0001_1_02_000000_0
2016-01-14 17:40:03,360 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793202490_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793202490_0001_1_02_000000_0
2016-01-14 17:40:03 Starting to run new task attempt: attempt_1452793202490_0001_1_02_000000_0
2016-01-14 17:40:03,360 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793202490_0001_1_02_000000_0
2016-01-14 17:40:03,361 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:03,362 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:03,362 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:03,363 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:03,363 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:03,363 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:03,363 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10] lower*upper/(1/1) ...lexcogrouprelativeunknown, VertexName: EA52DC3DE5634348B1A9EF657EF385CF, VertexParallelism: 1, TaskAttemptID:attempt_1452793202490_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E8968325287246B492DEEF5D5F88A88C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=AA2CEC34FD774094B4CE54270A5D62EC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=603769BFD376413B9E6575B09827A108, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:03,365 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:03,366 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793202490_0001_1_02_000000_0
2016-01-14 17:40:03,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:03,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:03,370 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793202490_0001_1_02_000000_0 sent events: (0-2)
2016-01-14 17:40:03,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E8968325287246B492DEEF5D5F88A88C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:03,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AA2CEC34FD774094B4CE54270A5D62EC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:03,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:03,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E8968325287246B492DEEF5D5F88A88C
2016-01-14 17:40:03,373 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:03,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AA2CEC34FD774094B4CE54270A5D62EC
2016-01-14 17:40:03,378 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:03,378 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:03,379 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E8968325287246B492DEEF5D5F88A88C
2016-01-14 17:40:03,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E8968325287246B492DEEF5D5F88A88C
2016-01-14 17:40:03,379 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:03,381 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:03,381 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:03,382 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: AA2CEC34FD774094B4CE54270A5D62EC
2016-01-14 17:40:03,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AA2CEC34FD774094B4CE54270A5D62EC
2016-01-14 17:40:03,384 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:03,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=603769BFD376413B9E6575B09827A108, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:03,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:03,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 603769BFD376413B9E6575B09827A108
2016-01-14 17:40:03,386 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:03,387 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 603769BFD376413B9E6575B09827A108
2016-01-14 17:40:03,396 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EA52DC3DE5634348B1A9EF657EF385CF, ordinal: 2
2016-01-14 17:40:03,396 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EA52DC3DE5634348B1A9EF657EF385CF, mem on start (mb), free: 319, total: 550, max: 672
2016-01-14 17:40:03,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:03,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-01-14 17:40:03,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:03,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:03,401 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:03,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 603769BFD376413B9E6575B09827A108
2016-01-14 17:40:03,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:03,402 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:03,402 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:03,402 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:03,402 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:03,403 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:03,403 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-01-14 17:40:03,403 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-01-14 17:40:03,403 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E8968325287246B492DEEF5D5F88A88C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 17:40:03,403 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AA2CEC34FD774094B4CE54270A5D62EC, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 17:40:03,404 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 603769BFD376413B9E6575B09827A108, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:03,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:03,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E8968325287246B492DEEF5D5F88A88C
2016-01-14 17:40:03,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E8968325287246B492DEEF5D5F88A88C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:03,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AA2CEC34FD774094B4CE54270A5D62EC
2016-01-14 17:40:03,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AA2CEC34FD774094B4CE54270A5D62EC being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:03,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-01-14 17:40:03,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-01-14 17:40:03,409 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:03,409 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:03,409 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:03,409 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:03,410 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E8968325287246B492DEEF5D5F88A88C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:03,410 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AA2CEC34FD774094B4CE54270A5D62EC with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:03,410 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 17:40:03,410 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 17:40:03,411 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:03,411 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:03,411 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:03,411 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:03,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E8968325287246B492DEEF5D5F88A88C
2016-01-14 17:40:03,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AA2CEC34FD774094B4CE54270A5D62EC
2016-01-14 17:40:03,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:03,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:03,417 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793202490_0001_1_02_000000_0
2016-01-14 17:40:03,431 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]]
2016-01-14 17:40:03,434 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793202490_0001_1_00_000000_0_10024, runDuration: 505592, ]
2016-01-14 17:40:03,435 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793202490_0001_1_01_000000_0_10027, runDuration: 507303, ]
2016-01-14 17:40:03,436 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] streamed: true, id: D2977B3A924740AB91DA576B49682034
2016-01-14 17:40:03,436 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793202490_0001_1_00_000000_0_10024] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452793203436, TimeTaken=1, Rate=0.09 MB/s
2016-01-14 17:40:03,437 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AA2CEC34FD774094B4CE54270A5D62EC
2016-01-14 17:40:03,436 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"], id: 603769BFD376413B9E6575B09827A108
2016-01-14 17:40:03,437 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:03,437 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AA2CEC34FD774094B4CE54270A5D62EC] #4 in 2ms
2016-01-14 17:40:03,437 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793202490_0001_1_01_000000_0_10027] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452793203437, TimeTaken=1, Rate=0.09 MB/s
2016-01-14 17:40:03,438 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E8968325287246B492DEEF5D5F88A88C
2016-01-14 17:40:03,438 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown"]
2016-01-14 17:40:03,438 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] D2977B3A924740AB91DA576B49682034, for 2 inputs
2016-01-14 17:40:03,439 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:-2] upper:[{1}:-2]] D2977B3A924740AB91DA576B49682034, for 2 inputs
2016-01-14 17:40:03,439 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:03,440 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E8968325287246B492DEEF5D5F88A88C] #5 in 4ms
2016-01-14 17:40:03,439 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AA2CEC34FD774094B4CE54270A5D62EC] #4, status:false, isInterrupted:false
2016-01-14 17:40:03,442 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:03,443 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 17:40:03,443 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:03,443 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:03,443 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 17:40:03,443 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AA2CEC34FD774094B4CE54270A5D62EC
2016-01-14 17:40:03,447 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:03,448 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E8968325287246B492DEEF5D5F88A88C] #5, status:false, isInterrupted:false
2016-01-14 17:40:03,448 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:03,449 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 17:40:03,449 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:03,449 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:03,449 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 17:40:03,450 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E8968325287246B492DEEF5D5F88A88C
2016-01-14 17:40:03,450 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:03,452 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 17:40:03,452 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 17:40:03,458 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EA52DC3DE5634348B1A9EF657EF385CF, all 2 inputs ready in: 00:00:00.000
2016-01-14 17:40:03,459 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1dd86227; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6913fdbf
2016-01-14 17:40:03,460 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@ecb8456; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@535d31b0
2016-01-14 17:40:03,466 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:03,466 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:03,467 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793202490_0001_1_02_000000_0
2016-01-14 17:40:03,468 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793202490_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 17:40:03,469 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793202490_0001_1_02_000000_0
2016-01-14 17:40:03,470 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EA52DC3DE5634348B1A9EF657EF385CF, mem on close (mb), free: 314, total: 550, max: 672
2016-01-14 17:40:03,470 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E8968325287246B492DEEF5D5F88A88C
2016-01-14 17:40:03,471 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:03,472 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:03,472 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:03,472 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:03,473 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:03,473 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AA2CEC34FD774094B4CE54270A5D62EC
2016-01-14 17:40:03,474 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:03,475 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:03,475 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:03,475 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:03,476 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:03 Completed running task attempt: attempt_1452793202490_0001_1_02_000000_0
2016-01-14 17:40:03,478 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793202490_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:03,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=31, MERGE_PHASE_TIME=41, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793203458, Process_Duration=11, Process_End_Time=1452793203469, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-01-14 17:40:03,479 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:03,480 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EA52DC3DE5634348B1A9EF657EF385CF, taskAttemptId=attempt_1452793202490_0001_1_02_000000_0, startTime=1452793203357, finishTime=1452793203479, timeTaken=122, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=31, MERGE_PHASE_TIME=41, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793203458, Process_Duration=11, Process_End_Time=1452793203469, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 17:40:03,480 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793202490_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:03,480 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:03,481 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:03,481 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:03,481 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793202490_0001_00_000003
2016-01-14 17:40:03,480 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793202490_0001_1_02_000000_0
2016-01-14 17:40:03,483 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:03,484 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:TASK_FINISHED]: vertexName=EA52DC3DE5634348B1A9EF657EF385CF, taskId=task_1452793202490_0001_1_02_000000, startTime=1452793203357, finishTime=1452793203483, timeTaken=126, status=SUCCEEDED, successfulAttemptID=attempt_1452793202490_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=31, MERGE_PHASE_TIME=41, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793203458, Process_Duration=11, Process_End_Time=1452793203469, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 17:40:03,484 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793202490_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:03,484 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF] : 1
2016-01-14 17:40:03,484 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF]
2016-01-14 17:40:03,485 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:03,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:VERTEX_FINISHED]: vertexName=EA52DC3DE5634348B1A9EF657EF385CF, vertexId=vertex_1452793202490_0001_1_02, initRequestedTime=1452793202701, initedTime=1452793202715, startRequestedTime=1452793202750, startedTime=1452793202750, finishTime=1452793203484, timeTaken=734, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=52, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=576716800, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=31, MERGE_PHASE_TIME=41, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793203458, Process_Duration=11, Process_End_Time=1452793203469, Read_Duration=1, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1452793203357, firstTasksToStart=[ task_1452793202490_0001_1_02_000000 ], lastTaskFinishTime=1452793203479, lastTasksToFinish=[ task_1452793202490_0001_1_02_000000 ], minTaskDuration=122, maxTaskDuration=122, avgTaskDuration=122.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793202490_0001_1_02_000000 ], longestDurationTasks=[ task_1452793202490_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:03,486 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:03,486 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:03,487 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793202490_0001_00_000003, stoppedTime=1452793203487, exitStatus=0
2016-01-14 17:40:03,487 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:03,487 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 17:40:03,487 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793202490_0001_1
2016-01-14 17:40:03,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:03,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793202490_0001_1
2016-01-14 17:40:03,489 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793202490_0001_1_00 [AA2CEC34FD774094B4CE54270A5D62EC]
2016-01-14 17:40:03,489 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 603769BFD376413B9E6575B09827A108 for vertex: vertex_1452793202490_0001_1_02 [EA52DC3DE5634348B1A9EF657EF385CF]
2016-01-14 17:40:03,492 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793202490_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 17:40:03,492 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793202490_0001_00_000003: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:03,492 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793202490_0001_00_000003, Exiting
2016-01-14 17:40:03,493 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793202490_0001_00_000003
2016-01-14 17:40:03,493 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793202490_0001_00_000003
2016-01-14 17:40:03,494 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793202490_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:03,494 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793202490_0001_00_000003 completed successfully
2016-01-14 17:40:03,500 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793202490_0001_1_01 [E8968325287246B492DEEF5D5F88A88C]
2016-01-14 17:40:03,504 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793202490_0001_1, queueSize=0
2016-01-14 17:40:03,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:03,505 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793202490_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793202490_0001_1, startTime=1452793202691, finishTime=1452793203500, timeTaken=809, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=459, FILE_BYTES_WRITTEN=11969, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=808, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1730150400, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=31, MERGE_PHASE_TIME=41, FIRST_EVENT_RECEIVED=26, LAST_EVENT_RECEIVED=26, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358379609631, Process_Duration=432, Process_End_Time=4358379610063, Read_Duration=420, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-01-14 17:40:03,505 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793202490_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:03,505 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793202490_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:03,506 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793202490_0001_00_000003
2016-01-14 17:40:03,508 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793202490_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:03 Completed Dag: dag_1452793202490_0001_1
2016-01-14 17:40:03,511 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793202490_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:03,511 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793202490_0001_1_post
2016-01-14 17:40:03,512 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:03,512 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:03,517 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lexcogrouprelativeunknown, applicationId=application_1452793202490_0001
2016-01-14 17:40:03,521 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:03,521 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:03,521 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:03,521 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:03,521 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:03,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:03,522 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:03,522 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:03,522 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:03,523 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:03,524 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:03,525 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:03,525 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:03,533 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BFA3C48192E4D64B32239A7FFCB163A/A6F156FCC8084410BDE2C0474CC65A10/.tez/application_1452793202490_0001
2016-01-14 17:40:03,534 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] completed in: 00:01.064
2016-01-14 17:40:03,614 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroupRelativeUnknown/complexcogrouprelativeunknown
2016-01-14 17:40:03,643 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:03,643 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:03,698 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 17:40:03,712 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-01-14 17:40:03,712 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:03,713 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:03,713 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:03,733 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 17:40:03,744 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-01-14 17:40:03,803 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 17:40:03,804 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:03,804 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-01-14 17:40:03,804 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 17:40:03,805 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 17:40:03,806 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 17:40:03,806 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...leRelative/simplerelative
2016-01-14 17:40:03,807 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:03,821 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 17:40:03,826 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:03,826 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:03,827 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:03,836 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033/.tez/application_1452793203826_0001 doesn't exist and is created
2016-01-14 17:40:03,852 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:03,852 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:03,853 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033/.tez/application_1452793203826_0001
2016-01-14 17:40:03,855 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793203826_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:03,867 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:03,869 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:03,871 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:03,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:03,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033/.tez/application_1452793203826_0001/localmode-log-dir
2016-01-14 17:40:03,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033/.tez/application_1452793203826_0001/localmode-log-dir/history.txt.appattempt_1452793203826_0001_000000, maxErrors=10
2016-01-14 17:40:03,873 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:03,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793203826_0001_000000, appSubmitTime=1452793203855, launchTime=1452793203855
2016-01-14 17:40:03,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:03,878 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:03,880 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:03,885 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:41319
2016-01-14 17:40:03,885 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:03,889 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:03,891 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793203826_0001_000000, startTime=1452793203891
2016-01-14 17:40:03,891 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:03,891 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:03,952 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:03,952 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:03,952 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1452793203826_0001, dagName=[1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033] test/(1/1) ...leRelative/simplerelative
2016-01-14 17:40:03,973 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033] test/(1/1) ...leRelative/simplerelative
2016-01-14 17:40:03,975 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793203826_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033/.tez/application_1452793203826_0001/localmode-log-dir/dag_1452793203826_0001_1.dot
2016-01-14 17:40:03,976 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033/.tez/application_1452793203826_0001/localmode-log-dir/dag_1452793203826_0001_1-tez-dag.pb.txt
2016-01-14 17:40:03,991 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793203826_0001_1
2016-01-14 17:40:03,991 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:03,991 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033] test/(1/1) ...leRelative/simplerelative
2016-01-14 17:40:03 Running Dag: dag_1452793203826_0001_1
2016-01-14 17:40:04,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793203826_0001_1, submitTime=1452793203973
2016-01-14 17:40:04,005 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A2494B7F29FA4C2BA0AB310951759C6E
2016-01-14 17:40:04,004 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:04,006 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:04,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:04,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793203826_0001_1, initTime=1452793204004
2016-01-14 17:40:04,008 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793203826_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:04,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:DAG_STARTED]: dagID=dag_1452793203826_0001_1, startTime=1452793204008
2016-01-14 17:40:04,008 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:04,008 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:04,009 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793203826_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:04,009 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A483C6AD087B46A7A6C3E45BAB0F8BE7 : {4826A748DEC54611AA3A29A984ADE3D4={InputName=4826A748DEC54611AA3A29A984ADE3D4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:04,009 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4826A748DEC54611AA3A29A984ADE3D4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:04,009 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7]
2016-01-14 17:40:04,009 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7]
2016-01-14 17:40:04,009 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7]
2016-01-14 17:40:04,010 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7]. Starting root input initializers: 1
2016-01-14 17:40:04,011 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:04,011 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4826A748DEC54611AA3A29A984ADE3D4 on vertex vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7]
2016-01-14 17:40:04,011 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E]
2016-01-14 17:40:04,153 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1452793203826_0001, dagName=[1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033] test/(1/1) ...leRelative/simplerelative
2016-01-14 17:40:04,154 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:04,155 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4826A748DEC54611AA3A29A984ADE3D4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:04,156 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:40:04,157 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E]
2016-01-14 17:40:04,157 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E]
2016-01-14 17:40:04,158 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E]
2016-01-14 17:40:04,158 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=93DA4F3071984649B2570FA5112BDB9F, vertexId=vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:04,158 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=93DA4F3071984649B2570FA5112BDB9F, vertexId=vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E]
2016-01-14 17:40:04,157 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:04,159 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:04,161 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=93DA4F3071984649B2570FA5112BDB9F, vertexName=A2494B7F29FA4C2BA0AB310951759C6E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:04,161 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=93DA4F3071984649B2570FA5112BDB9F, vertexId=vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E]
2016-01-14 17:40:04,161 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 17:40:04,162 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4826A748DEC54611AA3A29A984ADE3D4 on vertex vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7]
2016-01-14 17:40:04,163 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:04,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A2494B7F29FA4C2BA0AB310951759C6E, vertexId=vertex_1452793203826_0001_1_01, initRequestedTime=1452793204011, initedTime=1452793204162, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:40:04,163 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:40:04,163 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:04,164 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4826A748DEC54611AA3A29A984ADE3D4=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:04,164 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7] parallelism set to 1
2016-01-14 17:40:04,164 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:04,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793203826_0001_1_00, eventCount=1
2016-01-14 17:40:04,166 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:04,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A483C6AD087B46A7A6C3E45BAB0F8BE7, vertexId=vertex_1452793203826_0001_1_00, initRequestedTime=1452793204009, initedTime=1452793204165, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:04,167 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7] with distanceFromRoot: 0
2016-01-14 17:40:04,167 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:04,167 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A483C6AD087B46A7A6C3E45BAB0F8BE7
2016-01-14 17:40:04,168 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7]
2016-01-14 17:40:04,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793203826_0001_1_00, startRequestedTime=1452793204167, startedTime=1452793204167
2016-01-14 17:40:04,168 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:04,168 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:04,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:04,169 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:TASK_STARTED]: vertexName=A483C6AD087B46A7A6C3E45BAB0F8BE7, taskId=task_1452793203826_0001_1_00_000000, scheduledTime=1452793204169, launchTime=1452793204169
2016-01-14 17:40:04,170 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793203826_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:04,170 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793203826_0001_1_00 for vertex: vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E] numStartedSources: 1 numSources: 1
2016-01-14 17:40:04,170 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E] with distanceFromRoot: 1
2016-01-14 17:40:04,171 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7]
2016-01-14 17:40:04,171 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A483C6AD087B46A7A6C3E45BAB0F8BE7 in vertex: A2494B7F29FA4C2BA0AB310951759C6E
2016-01-14 17:40:04,171 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A2494B7F29FA4C2BA0AB310951759C6E with 1 source tasks and 1 pending tasks
2016-01-14 17:40:04,172 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:04,172 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793203826_0001_1_01, startRequestedTime=1452793204171, startedTime=1452793204171
2016-01-14 17:40:04,173 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:04,173 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033] test/(1/1) ...leRelative/simplerelative, VertexName: A483C6AD087B46A7A6C3E45BAB0F8BE7, VertexParallelism: 1, TaskAttemptID:attempt_1452793203826_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4826A748DEC54611AA3A29A984ADE3D4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A2494B7F29FA4C2BA0AB310951759C6E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:04,173 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793203826_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:04,174 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:04,174 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:04,175 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:04,175 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:04,176 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793203826_0001_00_000001
2016-01-14 17:40:04,176 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:04,177 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793203826_0001_00_000001
2016-01-14 17:40:04,177 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:04,178 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:04,178 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:04,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793203826_0001_00_000001, launchTime=1452793204178
2016-01-14 17:40:04,179 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793203826_0001_00_000001
2016-01-14 17:40:04,179 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793203826_0001_1_00_000000_0] to container: [container_1452793203826_0001_00_000001]
2016-01-14 17:40:04,179 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:04,180 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793203826_0001_00_000001 given task: attempt_1452793203826_0001_1_00_000000_0
2016-01-14 17:40:04,180 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793203826_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793203826_0001_1_00_000000_0
2016-01-14 17:40:04,180 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793203826_0001_1_00_000000_0] started. Is using containerId: [container_1452793203826_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:04,180 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452793203826_0001, with dag id: dag_1452793203826_0001_1
2016-01-14 17:40:04 Starting to run new task attempt: attempt_1452793203826_0001_1_00_000000_0
2016-01-14 17:40:04,180 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:04,181 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793203826_0001_1_00_000000_0
2016-01-14 17:40:04,180 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A483C6AD087B46A7A6C3E45BAB0F8BE7, taskAttemptId=attempt_1452793203826_0001_1_00_000000_0, startTime=1452793204180, containerId=container_1452793203826_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793203826_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:04,182 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793203826_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:04,182 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:04,183 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:04,183 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:04,183 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793203826_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:04,183 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:04,183 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:04,184 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:04,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033] test/(1/1) ...leRelative/simplerelative, VertexName: A483C6AD087B46A7A6C3E45BAB0F8BE7, VertexParallelism: 1, TaskAttemptID:attempt_1452793203826_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4826A748DEC54611AA3A29A984ADE3D4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A2494B7F29FA4C2BA0AB310951759C6E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:04,184 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:04,187 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793203826_0001_1_00_000000_0
2016-01-14 17:40:04,188 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793203826_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:04,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:04,188 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:04,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4826A748DEC54611AA3A29A984ADE3D4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:04,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:04,189 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A2494B7F29FA4C2BA0AB310951759C6E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:04,190 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:04,190 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A2494B7F29FA4C2BA0AB310951759C6E
2016-01-14 17:40:04,192 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:04,192 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A2494B7F29FA4C2BA0AB310951759C6E
2016-01-14 17:40:04,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A2494B7F29FA4C2BA0AB310951759C6E
2016-01-14 17:40:04,193 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:04,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:04,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4826A748DEC54611AA3A29A984ADE3D4
2016-01-14 17:40:04,194 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4826A748DEC54611AA3A29A984ADE3D4
2016-01-14 17:40:04,198 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:04,198 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4826A748DEC54611AA3A29A984ADE3D4
2016-01-14 17:40:04,198 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4826A748DEC54611AA3A29A984ADE3D4
2016-01-14 17:40:04,202 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A483C6AD087B46A7A6C3E45BAB0F8BE7, ordinal: 0
2016-01-14 17:40:04,202 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A483C6AD087B46A7A6C3E45BAB0F8BE7, mem on start (mb), free: 511, total: 542, max: 672
2016-01-14 17:40:04,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:04,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:04,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:04,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:04,202 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:04,203 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:04,203 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:04,203 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:04,203 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:04,203 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:04,203 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A2494B7F29FA4C2BA0AB310951759C6E, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:04,203 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4826A748DEC54611AA3A29A984ADE3D4, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:04,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:04,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4826A748DEC54611AA3A29A984ADE3D4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:04,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:04,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:04,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4826A748DEC54611AA3A29A984ADE3D4
2016-01-14 17:40:04,205 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:04,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4826A748DEC54611AA3A29A984ADE3D4
2016-01-14 17:40:04,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:04,206 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793203826_0001_1_00_000000_0
2016-01-14 17:40:04,206 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:04,207 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@9bcf17
2016-01-14 17:40:04,208 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@9bcf17
2016-01-14 17:40:04,209 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:04,209 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:04,211 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:04,211 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 4826A748DEC54611AA3A29A984ADE3D4
2016-01-14 17:40:04,212 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 0CA90B929B61403B9C75BF8185BD1CB5
2016-01-14 17:40:04,212 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 0CA90B929B61403B9C75BF8185BD1CB5
2016-01-14 17:40:04,212 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@12ae5e70; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@75ba14a4; comparator=cascading.tuple.hadoop.util.TupleComparator@ba1575e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:04,213 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:04,213 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:04,213 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:04,227 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:04,227 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:04,227 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:04,228 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:04,229 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:04,230 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:04,230 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:04,231 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A483C6AD087B46A7A6C3E45BAB0F8BE7, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:04,235 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793203826_0001_1_00_000000_0
2016-01-14 17:40:04,235 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A483C6AD087B46A7A6C3E45BAB0F8BE7, mem on close (mb), free: 409, total: 542, max: 672
2016-01-14 17:40:04,236 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:04,236 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:04,236 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-01-14 17:40:04,237 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-01-14 17:40:04,241 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:04 Completed running task attempt: attempt_1452793203826_0001_1_00_000000_0
2016-01-14 17:40:04,247 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793203826_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:04,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793203826_0001_1_00, eventCount=1
2016-01-14 17:40:04,249 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:04,250 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A483C6AD087B46A7A6C3E45BAB0F8BE7, taskAttemptId=attempt_1452793203826_0001_1_00_000000_0, startTime=1452793204180, finishTime=1452793204249, timeTaken=69, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793204232, Process_Duration=3, Process_End_Time=1452793204235, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 17:40:04,250 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793203826_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:04,250 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:04,250 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793203826_0001_1_00_000000_0
2016-01-14 17:40:04,251 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:04,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:TASK_FINISHED]: vertexName=A483C6AD087B46A7A6C3E45BAB0F8BE7, taskId=task_1452793203826_0001_1_00_000000, startTime=1452793204180, finishTime=1452793204250, timeTaken=70, status=SUCCEEDED, successfulAttemptID=attempt_1452793203826_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793204232, Process_Duration=3, Process_End_Time=1452793204235, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 17:40:04,251 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793203826_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:04,251 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7] : 1
2016-01-14 17:40:04,251 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7]
2016-01-14 17:40:04,255 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:04,255 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:VERTEX_FINISHED]: vertexName=A483C6AD087B46A7A6C3E45BAB0F8BE7, vertexId=vertex_1452793203826_0001_1_00, initRequestedTime=1452793204009, initedTime=1452793204165, startRequestedTime=1452793204167, startedTime=1452793204167, finishTime=1452793204251, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793204232, Process_Duration=3, Process_End_Time=1452793204235, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452793204180, firstTasksToStart=[ task_1452793203826_0001_1_00_000000 ], lastTaskFinishTime=1452793204249, lastTasksToFinish=[ task_1452793203826_0001_1_00_000000 ], minTaskDuration=69, maxTaskDuration=69, avgTaskDuration=69.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793203826_0001_1_00_000000 ], longestDurationTasks=[ task_1452793203826_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:04,255 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:04,256 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:04,256 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E] attempt: attempt_1452793203826_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:04,256 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A2494B7F29FA4C2BA0AB310951759C6E
2016-01-14 17:40:04,256 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E]
2016-01-14 17:40:04,256 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793203826_0001_00_000001, stoppedTime=1452793204256, exitStatus=0
2016-01-14 17:40:04,256 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:04,257 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:04,257 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:04,257 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:04,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793204232, Process_Duration=3, Process_End_Time=1452793204235, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 17:40:04,257 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:TASK_STARTED]: vertexName=A2494B7F29FA4C2BA0AB310951759C6E, taskId=task_1452793203826_0001_1_01_000000, scheduledTime=1452793204257, launchTime=1452793204257
2016-01-14 17:40:04,257 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793203826_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:04,258 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E]
2016-01-14 17:40:04,258 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793203826_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:40:04,258 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033] test/(1/1) ...leRelative/simplerelative, VertexName: A2494B7F29FA4C2BA0AB310951759C6E, VertexParallelism: 1, TaskAttemptID:attempt_1452793203826_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A483C6AD087B46A7A6C3E45BAB0F8BE7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=93DA4F3071984649B2570FA5112BDB9F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:04,258 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793203826_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:04,258 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:04,258 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:04,258 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:04,259 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:04,259 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793203826_0001_00_000001
2016-01-14 17:40:04,259 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793203826_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:04,259 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793203826_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:04,259 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793203826_0001_00_000001, Exiting
2016-01-14 17:40:04,260 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793203826_0001_00_000001
2016-01-14 17:40:04,260 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793203826_0001_00_000001
2016-01-14 17:40:04,260 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793203826_0001_00_000002
2016-01-14 17:40:04,260 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:04,263 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793203826_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:04,263 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793203826_0001_00_000001 completed successfully
2016-01-14 17:40:04,264 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:04,264 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793203826_0001_00_000002
2016-01-14 17:40:04,265 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:40:04,266 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:04,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:04,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793203826_0001_00_000002, launchTime=1452793204266
2016-01-14 17:40:04,267 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793203826_0001_00_000002
2016-01-14 17:40:04,268 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793203826_0001_1_01_000000_0] to container: [container_1452793203826_0001_00_000002]
2016-01-14 17:40:04,269 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:04,269 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793203826_0001_1_01_000000_0] started. Is using containerId: [container_1452793203826_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:40:04,269 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A2494B7F29FA4C2BA0AB310951759C6E, taskAttemptId=attempt_1452793203826_0001_1_01_000000_0, startTime=1452793204269, containerId=container_1452793203826_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793203826_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:40:04,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:04,270 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793203826_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:04,270 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793203826_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:04,270 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793203826_0001_00_000002 given task: attempt_1452793203826_0001_1_01_000000_0
2016-01-14 17:40:04,270 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793203826_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793203826_0001_1_01_000000_0
2016-01-14 17:40:04 Starting to run new task attempt: attempt_1452793203826_0001_1_01_000000_0
2016-01-14 17:40:04,271 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793203826_0001_1_01_000000_0
2016-01-14 17:40:04,271 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:04,271 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:04,271 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:04,271 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:04,271 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:04,272 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:04,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033] test/(1/1) ...leRelative/simplerelative, VertexName: A2494B7F29FA4C2BA0AB310951759C6E, VertexParallelism: 1, TaskAttemptID:attempt_1452793203826_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A483C6AD087B46A7A6C3E45BAB0F8BE7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=93DA4F3071984649B2570FA5112BDB9F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:04,272 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:04,272 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793203826_0001_1_01_000000_0
2016-01-14 17:40:04,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:04,273 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793203826_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:40:04,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:04,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:04,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=93DA4F3071984649B2570FA5112BDB9F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:04,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A483C6AD087B46A7A6C3E45BAB0F8BE7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:04,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:04,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:04,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A483C6AD087B46A7A6C3E45BAB0F8BE7
2016-01-14 17:40:04,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 93DA4F3071984649B2570FA5112BDB9F
2016-01-14 17:40:04,276 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:04,277 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 93DA4F3071984649B2570FA5112BDB9F
2016-01-14 17:40:04,278 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:04,278 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:04,279 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A483C6AD087B46A7A6C3E45BAB0F8BE7
2016-01-14 17:40:04,279 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:04,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A483C6AD087B46A7A6C3E45BAB0F8BE7
2016-01-14 17:40:04,288 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A2494B7F29FA4C2BA0AB310951759C6E, ordinal: 1
2016-01-14 17:40:04,288 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2494B7F29FA4C2BA0AB310951759C6E, mem on start (mb), free: 403, total: 542, max: 672
2016-01-14 17:40:04,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:04,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:04,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:04,288 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:04,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 93DA4F3071984649B2570FA5112BDB9F
2016-01-14 17:40:04,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:04,290 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:04,290 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:04,290 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:04,290 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:04,290 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:04,290 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:40:04,291 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 93DA4F3071984649B2570FA5112BDB9F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:04,291 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A483C6AD087B46A7A6C3E45BAB0F8BE7, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:40:04,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:04,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A483C6AD087B46A7A6C3E45BAB0F8BE7 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:04,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:04,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:04,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A483C6AD087B46A7A6C3E45BAB0F8BE7
2016-01-14 17:40:04,292 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:04,293 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:04,293 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A483C6AD087B46A7A6C3E45BAB0F8BE7 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:04,293 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:40:04,294 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:04,294 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:04,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A483C6AD087B46A7A6C3E45BAB0F8BE7
2016-01-14 17:40:04,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:04,298 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793203826_0001_1_01_000000_0
2016-01-14 17:40:04,300 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793203826_0001_1_00_000000_0_10033, runDuration: 54116, ]
2016-01-14 17:40:04,300 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 17:40:04,301 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 0CA90B929B61403B9C75BF8185BD1CB5
2016-01-14 17:40:04,301 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"], id: 93DA4F3071984649B2570FA5112BDB9F
2016-01-14 17:40:04,301 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative"]
2016-01-14 17:40:04,301 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 0CA90B929B61403B9C75BF8185BD1CB5, for 1 inputs
2016-01-14 17:40:04,302 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793203826_0001_1_00_000000_0_10033] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1452793204302, TimeTaken=2, Rate=0.10 MB/s
2016-01-14 17:40:04,302 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A483C6AD087B46A7A6C3E45BAB0F8BE7
2016-01-14 17:40:04,302 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:04,302 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A483C6AD087B46A7A6C3E45BAB0F8BE7] #6 in 2ms
2016-01-14 17:40:04,302 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A483C6AD087B46A7A6C3E45BAB0F8BE7] #6, status:false, isInterrupted:false
2016-01-14 17:40:04,307 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:04,307 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-01-14 17:40:04,308 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:04,308 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:04,308 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-01-14 17:40:04,308 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A483C6AD087B46A7A6C3E45BAB0F8BE7
2016-01-14 17:40:04,309 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A2494B7F29FA4C2BA0AB310951759C6E, all 1 inputs ready in: 00:00:00.006
2016-01-14 17:40:04,309 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:04,310 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3d57447e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2c51cba8
2016-01-14 17:40:04,312 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:04,312 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:04,313 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793203826_0001_1_01_000000_0
2016-01-14 17:40:04,313 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793203826_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:40:04,314 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793203826_0001_1_01_000000_0
2016-01-14 17:40:04,314 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2494B7F29FA4C2BA0AB310951759C6E, mem on close (mb), free: 400, total: 542, max: 672
2016-01-14 17:40:04,314 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A483C6AD087B46A7A6C3E45BAB0F8BE7
2016-01-14 17:40:04,315 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:04,315 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:04,315 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:04,315 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:04,316 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:04 Completed running task attempt: attempt_1452793203826_0001_1_01_000000_0
2016-01-14 17:40:04,316 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793203826_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:04,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793204309, Process_Duration=5, Process_End_Time=1452793204314, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-01-14 17:40:04,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A2494B7F29FA4C2BA0AB310951759C6E, taskAttemptId=attempt_1452793203826_0001_1_01_000000_0, startTime=1452793204269, finishTime=1452793204318, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793204309, Process_Duration=5, Process_End_Time=1452793204314, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 17:40:04,318 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:04,319 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793203826_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:04,319 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:04,319 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:04,319 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793203826_0001_1_01_000000_0
2016-01-14 17:40:04,320 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:04,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:04,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:TASK_FINISHED]: vertexName=A2494B7F29FA4C2BA0AB310951759C6E, taskId=task_1452793203826_0001_1_01_000000, startTime=1452793204269, finishTime=1452793204320, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1452793203826_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793204309, Process_Duration=5, Process_End_Time=1452793204314, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 17:40:04,321 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793203826_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:04,321 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E] : 1
2016-01-14 17:40:04,321 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E]
2016-01-14 17:40:04,322 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:04,323 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:VERTEX_FINISHED]: vertexName=A2494B7F29FA4C2BA0AB310951759C6E, vertexId=vertex_1452793203826_0001_1_01, initRequestedTime=1452793204011, initedTime=1452793204162, startRequestedTime=1452793204171, startedTime=1452793204171, finishTime=1452793204322, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793204309, Process_Duration=5, Process_End_Time=1452793204314, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1452793204269, firstTasksToStart=[ task_1452793203826_0001_1_01_000000 ], lastTaskFinishTime=1452793204318, lastTasksToFinish=[ task_1452793203826_0001_1_01_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793203826_0001_1_01_000000 ], longestDurationTasks=[ task_1452793203826_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:04,323 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:04,323 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:04,323 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793203826_0001_00_000002, stoppedTime=1452793204323, exitStatus=0
2016-01-14 17:40:04,323 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:04,323 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:04,323 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:04,324 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793203826_0001_00_000002
2016-01-14 17:40:04,324 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793203826_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:40:04,325 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793203826_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:04,325 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793203826_0001_00_000002, Exiting
2016-01-14 17:40:04,325 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793203826_0001_00_000002
2016-01-14 17:40:04,325 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793203826_0001_00_000002
2016-01-14 17:40:04,323 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793203826_0001_1
2016-01-14 17:40:04,326 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793203826_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:04,327 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793203826_0001_00_000002 completed successfully
2016-01-14 17:40:04,328 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793203826_0001_1
2016-01-14 17:40:04,328 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793203826_0001_1_00 [A483C6AD087B46A7A6C3E45BAB0F8BE7]
2016-01-14 17:40:04,328 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 93DA4F3071984649B2570FA5112BDB9F for vertex: vertex_1452793203826_0001_1_01 [A2494B7F29FA4C2BA0AB310951759C6E]
2016-01-14 17:40:04,340 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793203826_0001_1, queueSize=0
2016-01-14 17:40:04,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:04,341 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793203826_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793203826_0001_1, startTime=1452793204008, finishTime=1452793204337, timeTaken=329, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=376, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1137704960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905586408541, Process_Duration=8, Process_End_Time=2905586408549, Read_Duration=1, Tuples_Read=20, Tuples_Written=18, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-01-14 17:40:04,341 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793203826_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:04,341 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793203826_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:04,342 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793203826_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:04,343 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793203826_0001_00_000002
2016-01-14 17:40:04 Completed Dag: dag_1452793203826_0001_1
2016-01-14 17:40:04,345 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793203826_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:04,346 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793203826_0001_1_post
2016-01-14 17:40:04,346 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:04,346 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:04,350 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...leRelative/simplerelative, applicationId=application_1452793203826_0001
2016-01-14 17:40:04,352 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:04,353 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:04,353 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:04,353 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:04,353 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:04,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:04,354 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:04,354 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:04,354 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:04,354 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:04,355 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:04,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:04,356 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:04,360 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1A6CAA0843F24850A4A34C82BBB76CE2/C5923A8455804927A4530938DF449033/.tez/application_1452793203826_0001
2016-01-14 17:40:04,361 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.558
2016-01-14 17:40:04,399 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelative/simplerelative
2016-01-14 17:40:04,517 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:04,517 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:04,569 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-01-14 17:40:04,583 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.038
2016-01-14 17:40:04,584 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:04,584 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:04,584 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:04,601 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 17:40:04,616 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-01-14 17:40:04,668 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 17:40:04,669 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:04,669 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-01-14 17:40:04,669 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 17:40:04,670 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 17:40:04,670 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 17:40:04,671 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...SimpleResult/simpleresult
2016-01-14 17:40:04,671 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:04,684 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 17:40:04,688 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:04,689 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:04,689 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:04,698 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901/.tez/application_1452793204689_0001 doesn't exist and is created
2016-01-14 17:40:04,711 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:04,712 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:04,712 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901/.tez/application_1452793204689_0001
2016-01-14 17:40:04,714 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793204689_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:04,730 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:04,731 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:04,735 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:04,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:04,736 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901/.tez/application_1452793204689_0001/localmode-log-dir
2016-01-14 17:40:04,737 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901/.tez/application_1452793204689_0001/localmode-log-dir/history.txt.appattempt_1452793204689_0001_000000, maxErrors=10
2016-01-14 17:40:04,737 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:04,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793204689_0001_000000, appSubmitTime=1452793204714, launchTime=1452793204714
2016-01-14 17:40:04,742 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:04,743 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:04,744 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:04,744 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:04,747 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:56322
2016-01-14 17:40:04,750 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:04,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793204689_0001_000000, startTime=1452793204752
2016-01-14 17:40:04,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:04,752 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:04,812 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:04,812 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:04,812 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1452793204689_0001, dagName=[C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 17:40:04,831 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 17:40:04,833 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793204689_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901/.tez/application_1452793204689_0001/localmode-log-dir/dag_1452793204689_0001_1.dot
2016-01-14 17:40:04,833 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901/.tez/application_1452793204689_0001/localmode-log-dir/dag_1452793204689_0001_1-tez-dag.pb.txt
2016-01-14 17:40:04,850 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793204689_0001_1
2016-01-14 17:40:04,850 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:04,850 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 17:40:04 Running Dag: dag_1452793204689_0001_1
2016-01-14 17:40:04,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793204689_0001_1, submitTime=1452793204831
2016-01-14 17:40:04,860 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:04,861 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CBF548A983794176B125E52446EA193B
2016-01-14 17:40:04,861 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:04,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793204689_0001_1, initTime=1452793204860
2016-01-14 17:40:04,861 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:04,862 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793204689_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:04,862 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:DAG_STARTED]: dagID=dag_1452793204689_0001_1, startTime=1452793204862
2016-01-14 17:40:04,863 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:04,862 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:04,863 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793204689_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:04,863 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C06D43886B4C40F68CB24EAAE7F7CB3E : {E0BA225A9D674F71AAD28B2651F78E3E={InputName=E0BA225A9D674F71AAD28B2651F78E3E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:04,863 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E0BA225A9D674F71AAD28B2651F78E3E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:04,864 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E]
2016-01-14 17:40:04,864 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E]
2016-01-14 17:40:04,864 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E]
2016-01-14 17:40:04,864 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1452793204689_0001, dagName=[C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901] test/(1/1) ...SimpleResult/simpleresult
2016-01-14 17:40:04,864 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E]. Starting root input initializers: 1
2016-01-14 17:40:04,866 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:04,866 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B]
2016-01-14 17:40:04,867 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E0BA225A9D674F71AAD28B2651F78E3E on vertex vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E]
2016-01-14 17:40:04,869 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:04,869 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E0BA225A9D674F71AAD28B2651F78E3E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:04,871 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:04,871 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:40:04,872 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B]
2016-01-14 17:40:04,872 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B]
2016-01-14 17:40:04,872 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B]
2016-01-14 17:40:04,872 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=4D9439DB9F8647EBA332921BEA8B5FB4, vertexId=vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:04,872 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=4D9439DB9F8647EBA332921BEA8B5FB4, vertexId=vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B]
2016-01-14 17:40:04,872 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:04,874 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=4D9439DB9F8647EBA332921BEA8B5FB4, vertexName=CBF548A983794176B125E52446EA193B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:04,874 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=4D9439DB9F8647EBA332921BEA8B5FB4, vertexId=vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B]
2016-01-14 17:40:04,875 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:04,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CBF548A983794176B125E52446EA193B, vertexId=vertex_1452793204689_0001_1_01, initRequestedTime=1452793204866, initedTime=1452793204875, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:40:04,875 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:40:04,876 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 17:40:04,876 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E0BA225A9D674F71AAD28B2651F78E3E on vertex vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E]
2016-01-14 17:40:04,877 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:04,877 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E0BA225A9D674F71AAD28B2651F78E3E=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:04,877 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E] parallelism set to 1
2016-01-14 17:40:04,877 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:04,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793204689_0001_1_00, eventCount=1
2016-01-14 17:40:04,879 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:04,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C06D43886B4C40F68CB24EAAE7F7CB3E, vertexId=vertex_1452793204689_0001_1_00, initRequestedTime=1452793204863, initedTime=1452793204878, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:04,879 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E] with distanceFromRoot: 0
2016-01-14 17:40:04,879 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:04,880 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C06D43886B4C40F68CB24EAAE7F7CB3E
2016-01-14 17:40:04,880 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E]
2016-01-14 17:40:04,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793204689_0001_1_00, startRequestedTime=1452793204879, startedTime=1452793204879
2016-01-14 17:40:04,880 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:04,880 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:04,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:04,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:TASK_STARTED]: vertexName=C06D43886B4C40F68CB24EAAE7F7CB3E, taskId=task_1452793204689_0001_1_00_000000, scheduledTime=1452793204881, launchTime=1452793204881
2016-01-14 17:40:04,881 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793204689_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:04,882 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793204689_0001_1_00 for vertex: vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B] numStartedSources: 1 numSources: 1
2016-01-14 17:40:04,882 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B] with distanceFromRoot: 1
2016-01-14 17:40:04,882 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E]
2016-01-14 17:40:04,882 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C06D43886B4C40F68CB24EAAE7F7CB3E in vertex: CBF548A983794176B125E52446EA193B
2016-01-14 17:40:04,882 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CBF548A983794176B125E52446EA193B with 1 source tasks and 1 pending tasks
2016-01-14 17:40:04,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793204689_0001_1_01, startRequestedTime=1452793204882, startedTime=1452793204882
2016-01-14 17:40:04,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:04,883 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:04,883 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901] test/(1/1) ...SimpleResult/simpleresult, VertexName: C06D43886B4C40F68CB24EAAE7F7CB3E, VertexParallelism: 1, TaskAttemptID:attempt_1452793204689_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E0BA225A9D674F71AAD28B2651F78E3E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CBF548A983794176B125E52446EA193B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:04,884 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793204689_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:04,884 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:04,885 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:04,885 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:04,885 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:04,885 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793204689_0001_00_000001
2016-01-14 17:40:04,886 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:04,887 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793204689_0001_00_000001
2016-01-14 17:40:04,887 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:04,888 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:04,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:04,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793204689_0001_00_000001, launchTime=1452793204888
2016-01-14 17:40:04,889 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793204689_0001_00_000001
2016-01-14 17:40:04,890 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793204689_0001_1_00_000000_0] to container: [container_1452793204689_0001_00_000001]
2016-01-14 17:40:04,890 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:04,890 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793204689_0001_1_00_000000_0] started. Is using containerId: [container_1452793204689_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:04,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C06D43886B4C40F68CB24EAAE7F7CB3E, taskAttemptId=attempt_1452793204689_0001_1_00_000000_0, startTime=1452793204890, containerId=container_1452793204689_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793204689_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:04,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:04,891 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793204689_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:04,891 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793204689_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:04,891 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793204689_0001_00_000001 given task: attempt_1452793204689_0001_1_00_000000_0
2016-01-14 17:40:04,892 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793204689_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793204689_0001_1_00_000000_0
2016-01-14 17:40:04 Starting to run new task attempt: attempt_1452793204689_0001_1_00_000000_0
2016-01-14 17:40:04,892 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793204689_0001_1_00_000000_0
2016-01-14 17:40:04,893 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:04,893 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:04,893 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:04,893 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:04,893 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:04,894 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:04,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901] test/(1/1) ...SimpleResult/simpleresult, VertexName: C06D43886B4C40F68CB24EAAE7F7CB3E, VertexParallelism: 1, TaskAttemptID:attempt_1452793204689_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E0BA225A9D674F71AAD28B2651F78E3E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CBF548A983794176B125E52446EA193B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:04,894 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:04,895 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793204689_0001_1_00_000000_0
2016-01-14 17:40:04,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:04,895 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793204689_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:04,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:04,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E0BA225A9D674F71AAD28B2651F78E3E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:04,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CBF548A983794176B125E52446EA193B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:04,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:04,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:04,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:04,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E0BA225A9D674F71AAD28B2651F78E3E
2016-01-14 17:40:04,898 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E0BA225A9D674F71AAD28B2651F78E3E
2016-01-14 17:40:04,902 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:04,904 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452793204689_0001, with dag id: dag_1452793204689_0001_1
2016-01-14 17:40:04,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CBF548A983794176B125E52446EA193B
2016-01-14 17:40:04,906 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:04,907 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E0BA225A9D674F71AAD28B2651F78E3E
2016-01-14 17:40:04,907 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E0BA225A9D674F71AAD28B2651F78E3E
2016-01-14 17:40:04,911 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C06D43886B4C40F68CB24EAAE7F7CB3E, ordinal: 0
2016-01-14 17:40:04,912 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C06D43886B4C40F68CB24EAAE7F7CB3E, mem on start (mb), free: 502, total: 543, max: 672
2016-01-14 17:40:04,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:04,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:04,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:04,912 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:04,913 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CBF548A983794176B125E52446EA193B
2016-01-14 17:40:04,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CBF548A983794176B125E52446EA193B
2016-01-14 17:40:04,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:04,915 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:04,916 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:04,916 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:04,916 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:04,916 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:04,916 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:04,917 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E0BA225A9D674F71AAD28B2651F78E3E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:04,917 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CBF548A983794176B125E52446EA193B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:04,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:04,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E0BA225A9D674F71AAD28B2651F78E3E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:04,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E0BA225A9D674F71AAD28B2651F78E3E
2016-01-14 17:40:04,918 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:04,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E0BA225A9D674F71AAD28B2651F78E3E
2016-01-14 17:40:04,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:04,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:04,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:04,919 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793204689_0001_1_00_000000_0
2016-01-14 17:40:04,919 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:04,920 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4191eec5
2016-01-14 17:40:04,921 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4191eec5
2016-01-14 17:40:04,921 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:04,921 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:04,923 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:04,924 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: E0BA225A9D674F71AAD28B2651F78E3E
2016-01-14 17:40:04,925 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: 230264DB4FCA4453BD75BAE5AE9A9869
2016-01-14 17:40:04,925 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] 230264DB4FCA4453BD75BAE5AE9A9869
2016-01-14 17:40:04,926 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7391483c; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@471d966a; comparator=cascading.tuple.hadoop.util.TupleComparator@677e2d9f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:04,926 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:04,926 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:04,927 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:04,940 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:04,940 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:04,940 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:04,940 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:04,941 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:05,015 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:05,016 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:05,016 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C06D43886B4C40F68CB24EAAE7F7CB3E, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:05,019 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:40:05,019 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:40:05,020 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793204689_0001_1_00_000000_0
2016-01-14 17:40:05,020 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C06D43886B4C40F68CB24EAAE7F7CB3E, mem on close (mb), free: 395, total: 543, max: 672
2016-01-14 17:40:05,020 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:05,020 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:05,021 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 22; bufvoid = 104857600
2016-01-14 17:40:05,021 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600
2016-01-14 17:40:05,022 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:05 Completed running task attempt: attempt_1452793204689_0001_1_00_000000_0
2016-01-14 17:40:05,026 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793204689_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:05,026 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793204689_0001_1_00, eventCount=1
2016-01-14 17:40:05,026 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793205016, Process_Duration=4, Process_End_Time=1452793205020, Read_Duration=3, Tuples_Read=5, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 17:40:05,027 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:05,027 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:05,027 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:05,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C06D43886B4C40F68CB24EAAE7F7CB3E, taskAttemptId=attempt_1452793204689_0001_1_00_000000_0, startTime=1452793204890, finishTime=1452793205027, timeTaken=137, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793205016, Process_Duration=4, Process_End_Time=1452793205020, Read_Duration=3, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:05,028 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793204689_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:05,028 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793204689_0001_00_000001
2016-01-14 17:40:05,028 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:05,028 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793204689_0001_1_00_000000_0
2016-01-14 17:40:05,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:05,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:TASK_FINISHED]: vertexName=C06D43886B4C40F68CB24EAAE7F7CB3E, taskId=task_1452793204689_0001_1_00_000000, startTime=1452793204890, finishTime=1452793205029, timeTaken=139, status=SUCCEEDED, successfulAttemptID=attempt_1452793204689_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793205016, Process_Duration=4, Process_End_Time=1452793205020, Read_Duration=3, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:05,030 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793204689_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:05,030 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E] : 1
2016-01-14 17:40:05,030 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E]
2016-01-14 17:40:05,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:05,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:VERTEX_FINISHED]: vertexName=C06D43886B4C40F68CB24EAAE7F7CB3E, vertexId=vertex_1452793204689_0001_1_00, initRequestedTime=1452793204863, initedTime=1452793204878, startRequestedTime=1452793204879, startedTime=1452793204879, finishTime=1452793205030, timeTaken=151, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=68, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=2, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=2, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793205016, Process_Duration=4, Process_End_Time=1452793205020, Read_Duration=3, Tuples_Read=5, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452793204890, firstTasksToStart=[ task_1452793204689_0001_1_00_000000 ], lastTaskFinishTime=1452793205027, lastTasksToFinish=[ task_1452793204689_0001_1_00_000000 ], minTaskDuration=137, maxTaskDuration=137, avgTaskDuration=137.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793204689_0001_1_00_000000 ], longestDurationTasks=[ task_1452793204689_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:05,032 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:05,032 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:05,032 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:05,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793204689_0001_00_000001, stoppedTime=1452793205032, exitStatus=0
2016-01-14 17:40:05,032 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:05,033 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B] attempt: attempt_1452793204689_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:05,033 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CBF548A983794176B125E52446EA193B
2016-01-14 17:40:05,033 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B]
2016-01-14 17:40:05,033 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:05,033 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793204689_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:40:05,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:05,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:TASK_STARTED]: vertexName=CBF548A983794176B125E52446EA193B, taskId=task_1452793204689_0001_1_01_000000, scheduledTime=1452793205034, launchTime=1452793205034
2016-01-14 17:40:05,034 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793204689_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:05,035 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:05,035 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B]
2016-01-14 17:40:05,035 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901] test/(1/1) ...SimpleResult/simpleresult, VertexName: CBF548A983794176B125E52446EA193B, VertexParallelism: 1, TaskAttemptID:attempt_1452793204689_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C06D43886B4C40F68CB24EAAE7F7CB3E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4D9439DB9F8647EBA332921BEA8B5FB4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:05,035 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793204689_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:05,035 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:05,036 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793204689_0001_00_000002
2016-01-14 17:40:05,037 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:05,038 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793204689_0001_00_000002
2016-01-14 17:40:05,038 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:05,039 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793204689_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:05,039 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:05,039 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793204689_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:05,040 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793204689_0001_00_000001, Exiting
2016-01-14 17:40:05,040 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793204689_0001_00_000001
2016-01-14 17:40:05,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793204689_0001_00_000002, launchTime=1452793205039
2016-01-14 17:40:05,040 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:05,040 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793204689_0001_00_000001
2016-01-14 17:40:05,041 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793204689_0001_00_000002
2016-01-14 17:40:05,043 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793204689_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:05,044 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793204689_0001_00_000001 completed successfully
2016-01-14 17:40:05,044 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:05,044 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793204689_0001_1_01_000000_0] to container: [container_1452793204689_0001_00_000002]
2016-01-14 17:40:05,044 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:05,045 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793204689_0001_00_000002 given task: attempt_1452793204689_0001_1_01_000000_0
2016-01-14 17:40:05,045 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793204689_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793204689_0001_1_01_000000_0
2016-01-14 17:40:05,045 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793204689_0001_1_01_000000_0] started. Is using containerId: [container_1452793204689_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:40:05 Starting to run new task attempt: attempt_1452793204689_0001_1_01_000000_0
2016-01-14 17:40:05,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CBF548A983794176B125E52446EA193B, taskAttemptId=attempt_1452793204689_0001_1_01_000000_0, startTime=1452793205045, containerId=container_1452793204689_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793204689_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:40:05,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:05,045 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793204689_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:05,045 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793204689_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:05,045 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793204689_0001_1_01_000000_0
2016-01-14 17:40:05,046 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:05,046 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:05,046 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:05,046 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:05,046 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:05,046 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:05,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901] test/(1/1) ...SimpleResult/simpleresult, VertexName: CBF548A983794176B125E52446EA193B, VertexParallelism: 1, TaskAttemptID:attempt_1452793204689_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C06D43886B4C40F68CB24EAAE7F7CB3E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4D9439DB9F8647EBA332921BEA8B5FB4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:05,047 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:05,050 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793204689_0001_1_01_000000_0
2016-01-14 17:40:05,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:05,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:05,050 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793204689_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:40:05,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:05,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C06D43886B4C40F68CB24EAAE7F7CB3E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:05,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=4D9439DB9F8647EBA332921BEA8B5FB4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:05,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:05,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:05,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C06D43886B4C40F68CB24EAAE7F7CB3E
2016-01-14 17:40:05,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 4D9439DB9F8647EBA332921BEA8B5FB4
2016-01-14 17:40:05,053 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:05,054 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 4D9439DB9F8647EBA332921BEA8B5FB4
2016-01-14 17:40:05,055 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:05,055 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:05,055 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C06D43886B4C40F68CB24EAAE7F7CB3E
2016-01-14 17:40:05,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C06D43886B4C40F68CB24EAAE7F7CB3E
2016-01-14 17:40:05,057 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:05,065 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:05,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 4D9439DB9F8647EBA332921BEA8B5FB4
2016-01-14 17:40:05,067 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CBF548A983794176B125E52446EA193B, ordinal: 1
2016-01-14 17:40:05,068 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CBF548A983794176B125E52446EA193B, mem on start (mb), free: 390, total: 543, max: 672
2016-01-14 17:40:05,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:05,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:05,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:05,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:05,069 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:05,069 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:05,069 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:05,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:05,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:05,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:40:05,071 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 4D9439DB9F8647EBA332921BEA8B5FB4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:05,071 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C06D43886B4C40F68CB24EAAE7F7CB3E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:40:05,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:05,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C06D43886B4C40F68CB24EAAE7F7CB3E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:05,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C06D43886B4C40F68CB24EAAE7F7CB3E
2016-01-14 17:40:05,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:05,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:05,072 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:05,073 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:05,074 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C06D43886B4C40F68CB24EAAE7F7CB3E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:05,074 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:40:05,074 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:05,074 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:05,077 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C06D43886B4C40F68CB24EAAE7F7CB3E
2016-01-14 17:40:05,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:05,078 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793204689_0001_1_01_000000_0
2016-01-14 17:40:05,079 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793204689_0001_1_00_000000_0_10040, runDuration: 118325, ]
2016-01-14 17:40:05,080 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793204689_0001_1_00_000000_0_10040] to DISK_DIRECT, CompressedSize=36, DecompressedSize=32,EndTime=1452793205080, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:40:05,081 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C06D43886B4C40F68CB24EAAE7F7CB3E
2016-01-14 17:40:05,081 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:05,081 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C06D43886B4C40F68CB24EAAE7F7CB3E] #7 in 2ms
2016-01-14 17:40:05,081 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C06D43886B4C40F68CB24EAAE7F7CB3E] #7, status:false, isInterrupted:false
2016-01-14 17:40:05,082 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:05,082 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 36 bytes from disk
2016-01-14 17:40:05,082 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:05,082 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:05,083 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 17 bytes
2016-01-14 17:40:05,083 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C06D43886B4C40F68CB24EAAE7F7CB3E
2016-01-14 17:40:05,083 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:05,084 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 17:40:05,084 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: 230264DB4FCA4453BD75BAE5AE9A9869
2016-01-14 17:40:05,084 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"], id: 4D9439DB9F8647EBA332921BEA8B5FB4
2016-01-14 17:40:05,085 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult"]
2016-01-14 17:40:05,085 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] 230264DB4FCA4453BD75BAE5AE9A9869, for 1 inputs
2016-01-14 17:40:05,085 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CBF548A983794176B125E52446EA193B, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:05,086 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@52ee705c; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2412d28d
2016-01-14 17:40:05,087 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:05,087 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:05,088 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793204689_0001_1_01_000000_0
2016-01-14 17:40:05,088 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793204689_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:40:05,089 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793204689_0001_1_01_000000_0
2016-01-14 17:40:05,090 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CBF548A983794176B125E52446EA193B, mem on close (mb), free: 387, total: 543, max: 672
2016-01-14 17:40:05,090 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C06D43886B4C40F68CB24EAAE7F7CB3E
2016-01-14 17:40:05,090 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:05,090 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:05,090 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:05,091 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:05,091 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:05 Completed running task attempt: attempt_1452793204689_0001_1_01_000000_0
2016-01-14 17:40:05,091 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793204689_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:05,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793205085, Process_Duration=4, Process_End_Time=1452793205089, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=2]]
2016-01-14 17:40:05,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CBF548A983794176B125E52446EA193B, taskAttemptId=attempt_1452793204689_0001_1_01_000000_0, startTime=1452793205045, finishTime=1452793205092, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793205085, Process_Duration=4, Process_End_Time=1452793205089, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-01-14 17:40:05,093 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793204689_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:05,093 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:05,093 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:05,093 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:05,093 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793204689_0001_1_01_000000_0
2016-01-14 17:40:05,093 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:05,094 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793204689_0001_00_000002
2016-01-14 17:40:05,094 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:05,094 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:TASK_FINISHED]: vertexName=CBF548A983794176B125E52446EA193B, taskId=task_1452793204689_0001_1_01_000000, startTime=1452793205045, finishTime=1452793205093, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1452793204689_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793205085, Process_Duration=4, Process_End_Time=1452793205089, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2
2016-01-14 17:40:05,094 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793204689_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:05,094 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B] : 1
2016-01-14 17:40:05,095 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B]
2016-01-14 17:40:05,096 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:05,096 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:VERTEX_FINISHED]: vertexName=CBF548A983794176B125E52446EA193B, vertexId=vertex_1452793204689_0001_1_01, initRequestedTime=1452793204866, initedTime=1452793204875, startRequestedTime=1452793204882, startedTime=1452793204882, finishTime=1452793205095, timeTaken=213, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=68, FILE_BYTES_WRITTEN=24, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=2, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793205085, Process_Duration=4, Process_End_Time=1452793205089, Read_Duration=0, Tuples_Read=2, Tuples_Written=2, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=2, vertexStats=firstTaskStartTime=1452793205045, firstTasksToStart=[ task_1452793204689_0001_1_01_000000 ], lastTaskFinishTime=1452793205092, lastTasksToFinish=[ task_1452793204689_0001_1_01_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793204689_0001_1_01_000000 ], longestDurationTasks=[ task_1452793204689_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:05,096 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:05,096 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:05,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793204689_0001_00_000002, stoppedTime=1452793205097, exitStatus=0
2016-01-14 17:40:05,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:05,097 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:05,097 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:05,097 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793204689_0001_1
2016-01-14 17:40:05,098 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793204689_0001_1
2016-01-14 17:40:05,098 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 4D9439DB9F8647EBA332921BEA8B5FB4 for vertex: vertex_1452793204689_0001_1_01 [CBF548A983794176B125E52446EA193B]
2016-01-14 17:40:05,104 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793204689_0001_1_00 [C06D43886B4C40F68CB24EAAE7F7CB3E]
2016-01-14 17:40:05,105 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793204689_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:40:05,105 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793204689_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:05,105 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793204689_0001_00_000002, Exiting
2016-01-14 17:40:05,105 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793204689_0001_00_000002
2016-01-14 17:40:05,105 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793204689_0001_00_000002
2016-01-14 17:40:05,105 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793204689_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:05,105 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793204689_0001_00_000002 completed successfully
2016-01-14 17:40:05,107 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793204689_0001_1, queueSize=0
2016-01-14 17:40:05,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:05,109 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793204689_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793204689_0001_1, startTime=1452793204862, finishTime=1452793205104, timeTaken=242, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=130, FILE_BYTES_WRITTEN=92, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=1, REDUCE_INPUT_RECORDS=2, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=4, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1139802112, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=4, OUTPUT_BYTES=22, OUTPUT_BYTES_WITH_OVERHEAD=32, OUTPUT_BYTES_PHYSICAL=36, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=36, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=36, SHUFFLE_BYTES_DECOMPRESSED=32, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=36, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905586410101, Process_Duration=8, Process_End_Time=2905586410109, Read_Duration=3, Tuples_Read=7, Tuples_Written=4, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=2
2016-01-14 17:40:05,109 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793204689_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:05,109 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793204689_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:05,109 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793204689_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:05,109 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793204689_0001_00_000002
2016-01-14 17:40:05 Completed Dag: dag_1452793204689_0001_1
2016-01-14 17:40:05,112 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793204689_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:05,113 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793204689_0001_1_post
2016-01-14 17:40:05,113 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:05,113 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:05,116 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...SimpleResult/simpleresult, applicationId=application_1452793204689_0001
2016-01-14 17:40:05,119 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:05,119 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:05,119 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:05,119 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:05,119 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:05,119 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:05,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:05,121 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:05,121 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:05,121 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:05,122 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:05,122 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:05,122 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:05,127 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C0DC434BFFA94828A85A6D55457D7D54/D15FCB4670BB4821911DF929431FB901/.tez/application_1452793204689_0001
2016-01-14 17:40:05,128 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.459
2016-01-14 17:40:05,165 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleResult/simpleresult
2016-01-14 17:40:05,191 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:05,192 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:05,236 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 17:40:05,241 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-01-14 17:40:05,242 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 17:40:05,242 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 17:40:05,242 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:05,259 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 17:40:05,259 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-01-14 17:40:05,269 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 17:40:05,269 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:05,270 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-01-14 17:40:05,270 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 17:40:05,270 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 17:40:05,270 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 17:40:05,271 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...m/testFilterAll/filterall
2016-01-14 17:40:05,271 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:05,285 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 17:40:05,289 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:05,290 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:05,290 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:05,298 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2/.tez/application_1452793205289_0001 doesn't exist and is created
2016-01-14 17:40:05,312 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:05,312 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:05,313 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2/.tez/application_1452793205289_0001
2016-01-14 17:40:05,316 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793205289_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:05,328 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:05,328 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:05,330 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:05,330 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:05,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2/.tez/application_1452793205289_0001/localmode-log-dir
2016-01-14 17:40:05,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2/.tez/application_1452793205289_0001/localmode-log-dir/history.txt.appattempt_1452793205289_0001_000000, maxErrors=10
2016-01-14 17:40:05,330 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:05,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793205289_0001_000000, appSubmitTime=1452793205316, launchTime=1452793205316
2016-01-14 17:40:05,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:05,338 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:05,338 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:05,338 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:05,340 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:48550
2016-01-14 17:40:05,342 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:05,344 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793205289_0001_000000, startTime=1452793205344
2016-01-14 17:40:05,344 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:05,344 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:05,413 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:05,413 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:05,413 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1452793205289_0001, dagName=[A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 17:40:05,430 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 17:40:05,432 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793205289_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2/.tez/application_1452793205289_0001/localmode-log-dir/dag_1452793205289_0001_1.dot
2016-01-14 17:40:05,433 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2/.tez/application_1452793205289_0001/localmode-log-dir/dag_1452793205289_0001_1-tez-dag.pb.txt
2016-01-14 17:40:05,441 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793205289_0001_1
2016-01-14 17:40:05,441 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:05,441 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 17:40:05 Running Dag: dag_1452793205289_0001_1
2016-01-14 17:40:05,450 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793205289_0001_1, submitTime=1452793205431
2016-01-14 17:40:05,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:05,451 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A8938BDF989E48F181834C15E41534C1
2016-01-14 17:40:05,451 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:05,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793205289_0001_1, initTime=1452793205450
2016-01-14 17:40:05,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:05,451 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793205289_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:05,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:DAG_STARTED]: dagID=dag_1452793205289_0001_1, startTime=1452793205452
2016-01-14 17:40:05,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:05,453 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:05,454 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793205289_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:05,454 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A8938BDF989E48F181834C15E41534C1 : {451905200B6D41F88DAF4A9CE9C19781={InputName=451905200B6D41F88DAF4A9CE9C19781}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:05,454 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 451905200B6D41F88DAF4A9CE9C19781, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:05,454 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,454 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,454 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,455 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]. Starting root input initializers: 1
2016-01-14 17:40:05,454 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1452793205289_0001, dagName=[A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2] test/(1/1) ...m/testFilterAll/filterall
2016-01-14 17:40:05,456 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:05,456 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 451905200B6D41F88DAF4A9CE9C19781 on vertex vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,457 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:05,458 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 451905200B6D41F88DAF4A9CE9C19781 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:05,458 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:05,459 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:05,460 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 17:40:05,461 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 451905200B6D41F88DAF4A9CE9C19781 on vertex vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,461 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:05,461 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {451905200B6D41F88DAF4A9CE9C19781=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:05,461 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1] parallelism set to 1
2016-01-14 17:40:05,461 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:05,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793205289_0001_1_00, eventCount=1
2016-01-14 17:40:05,462 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,462 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B14F78BAE690492EAD7F33E183E87ECD, vertexId=vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:05,462 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B14F78BAE690492EAD7F33E183E87ECD, vertexId=vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,464 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B14F78BAE690492EAD7F33E183E87ECD, vertexName=A8938BDF989E48F181834C15E41534C1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:05,464 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B14F78BAE690492EAD7F33E183E87ECD, vertexId=vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A8938BDF989E48F181834C15E41534C1, vertexId=vertex_1452793205289_0001_1_00, initRequestedTime=1452793205454, initedTime=1452793205465, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:05,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:05,465 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1] with distanceFromRoot: 0
2016-01-14 17:40:05,465 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:05,466 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A8938BDF989E48F181834C15E41534C1
2016-01-14 17:40:05,466 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,467 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793205289_0001_1_00, startRequestedTime=1452793205465, startedTime=1452793205465
2016-01-14 17:40:05,467 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:05,467 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:05,468 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:05,468 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:TASK_STARTED]: vertexName=A8938BDF989E48F181834C15E41534C1, taskId=task_1452793205289_0001_1_00_000000, scheduledTime=1452793205467, launchTime=1452793205467
2016-01-14 17:40:05,468 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793205289_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:05,468 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,468 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2] test/(1/1) ...m/testFilterAll/filterall, VertexName: A8938BDF989E48F181834C15E41534C1, VertexParallelism: 1, TaskAttemptID:attempt_1452793205289_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=451905200B6D41F88DAF4A9CE9C19781, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B14F78BAE690492EAD7F33E183E87ECD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:05,468 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793205289_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:05,469 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:05,469 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:05,470 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:05,470 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:05,470 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793205289_0001_00_000001
2016-01-14 17:40:05,470 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205289_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:05,472 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793205289_0001_00_000001
2016-01-14 17:40:05,472 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:05,473 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205289_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:05,473 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:05,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793205289_0001_00_000001, launchTime=1452793205473
2016-01-14 17:40:05,474 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793205289_0001_00_000001
2016-01-14 17:40:05,475 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793205289_0001_1_00_000000_0] to container: [container_1452793205289_0001_00_000001]
2016-01-14 17:40:05,475 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205289_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:05,475 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793205289_0001_1_00_000000_0] started. Is using containerId: [container_1452793205289_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:05,476 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:05,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A8938BDF989E48F181834C15E41534C1, taskAttemptId=attempt_1452793205289_0001_1_00_000000_0, startTime=1452793205475, containerId=container_1452793205289_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793205289_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:05,476 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793205289_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:05,476 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793205289_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:05,477 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793205289_0001_00_000001 given task: attempt_1452793205289_0001_1_00_000000_0
2016-01-14 17:40:05,477 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793205289_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793205289_0001_1_00_000000_0
2016-01-14 17:40:05 Starting to run new task attempt: attempt_1452793205289_0001_1_00_000000_0
2016-01-14 17:40:05,477 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793205289_0001_1_00_000000_0
2016-01-14 17:40:05,477 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:05,478 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:05,478 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:05,478 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:05,478 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:05,478 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:05,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2] test/(1/1) ...m/testFilterAll/filterall, VertexName: A8938BDF989E48F181834C15E41534C1, VertexParallelism: 1, TaskAttemptID:attempt_1452793205289_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=451905200B6D41F88DAF4A9CE9C19781, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B14F78BAE690492EAD7F33E183E87ECD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:05,478 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:05,479 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793205289_0001_1_00_000000_0
2016-01-14 17:40:05,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:05,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:05,479 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452793205289_0001, with dag id: dag_1452793205289_0001_1
2016-01-14 17:40:05,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=451905200B6D41F88DAF4A9CE9C19781, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:05,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:05,486 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:05,489 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793205289_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:05,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B14F78BAE690492EAD7F33E183E87ECD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:05,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:05,490 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B14F78BAE690492EAD7F33E183E87ECD
2016-01-14 17:40:05,490 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:05,490 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B14F78BAE690492EAD7F33E183E87ECD
2016-01-14 17:40:05,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:05,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 451905200B6D41F88DAF4A9CE9C19781
2016-01-14 17:40:05,495 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 451905200B6D41F88DAF4A9CE9C19781
2016-01-14 17:40:05,498 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:05,498 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 451905200B6D41F88DAF4A9CE9C19781
2016-01-14 17:40:05,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 451905200B6D41F88DAF4A9CE9C19781
2016-01-14 17:40:05,500 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A8938BDF989E48F181834C15E41534C1, ordinal: 0
2016-01-14 17:40:05,501 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8938BDF989E48F181834C15E41534C1, mem on start (mb), free: 308, total: 543, max: 672
2016-01-14 17:40:05,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:05,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:05,502 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:05,503 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:05,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B14F78BAE690492EAD7F33E183E87ECD
2016-01-14 17:40:05,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:05,503 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:05,504 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:05,504 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:05,504 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:05,505 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:05,505 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:05,505 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 17:40:05,506 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B14F78BAE690492EAD7F33E183E87ECD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:05,506 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 451905200B6D41F88DAF4A9CE9C19781, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:05,506 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:05,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 451905200B6D41F88DAF4A9CE9C19781
2016-01-14 17:40:05,507 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:05,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 451905200B6D41F88DAF4A9CE9C19781
2016-01-14 17:40:05,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 451905200B6D41F88DAF4A9CE9C19781 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:05,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:05,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:05,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:05,509 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793205289_0001_1_00_000000_0
2016-01-14 17:40:05,511 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:05,511 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:05,512 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 451905200B6D41F88DAF4A9CE9C19781
2016-01-14 17:40:05,512 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"], id: B14F78BAE690492EAD7F33E183E87ECD
2016-01-14 17:40:05,512 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilterAll/filterall"]
2016-01-14 17:40:05,513 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6f29e045
2016-01-14 17:40:05,513 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:05,513 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:05,514 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6f29e045
2016-01-14 17:40:05,514 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:05,514 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:05,516 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A8938BDF989E48F181834C15E41534C1, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:05,518 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:05,518 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:05,518 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793205289_0001_1_00_000000_0
2016-01-14 17:40:05,518 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793205289_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 17:40:05,519 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793205289_0001_1_00_000000_0
2016-01-14 17:40:05,519 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A8938BDF989E48F181834C15E41534C1, mem on close (mb), free: 306, total: 543, max: 672
2016-01-14 17:40:05 Completed running task attempt: attempt_1452793205289_0001_1_00_000000_0
2016-01-14 17:40:05,519 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793205289_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:05,521 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:05,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A8938BDF989E48F181834C15E41534C1, taskAttemptId=attempt_1452793205289_0001_1_00_000000_0, startTime=1452793205475, finishTime=1452793205521, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793205516, Process_Duration=3, Process_End_Time=1452793205519, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 17:40:05,522 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793205289_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:05,522 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:05,522 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793205289_0001_1_00_000000_0
2016-01-14 17:40:05,523 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:05,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:TASK_FINISHED]: vertexName=A8938BDF989E48F181834C15E41534C1, taskId=task_1452793205289_0001_1_00_000000, startTime=1452793205475, finishTime=1452793205522, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1452793205289_0001_1_00_000000_0, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793205516, Process_Duration=3, Process_End_Time=1452793205519, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 17:40:05,523 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793205289_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:05,523 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1] : 1
2016-01-14 17:40:05,524 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,524 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:05,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:VERTEX_FINISHED]: vertexName=A8938BDF989E48F181834C15E41534C1, vertexId=vertex_1452793205289_0001_1_00, initRequestedTime=1452793205454, initedTime=1452793205465, startRequestedTime=1452793205465, startedTime=1452793205465, finishTime=1452793205524, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 19, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793205516, Process_Duration=3, Process_End_Time=1452793205519, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452793205475, firstTasksToStart=[ task_1452793205289_0001_1_00_000000 ], lastTaskFinishTime=1452793205521, lastTasksToFinish=[ task_1452793205289_0001_1_00_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793205289_0001_1_00_000000 ], longestDurationTasks=[ task_1452793205289_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:05,534 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:05,534 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205289_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:05,534 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793205289_0001_00_000001, stoppedTime=1452793205534, exitStatus=0
2016-01-14 17:40:05,535 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:05,535 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205289_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:05,535 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 17:40:05,535 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793205289_0001_1
2016-01-14 17:40:05,536 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793205289_0001_1
2016-01-14 17:40:05,536 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: B14F78BAE690492EAD7F33E183E87ECD for vertex: vertex_1452793205289_0001_1_00 [A8938BDF989E48F181834C15E41534C1]
2016-01-14 17:40:05,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 19 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0][cascading.flow.SliceCounters Process_Begin_Time=1452793205516, Process_Duration=3, Process_End_Time=1452793205519, Read_Duration=1, Tuples_Read=10][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 17:40:05,537 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:05,537 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:05,538 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793205289_0001_00_000001
2016-01-14 17:40:05,538 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793205289_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:05,538 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793205289_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:05,538 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793205289_0001_00_000001, Exiting
2016-01-14 17:40:05,538 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793205289_0001_00_000001
2016-01-14 17:40:05,538 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793205289_0001_00_000001
2016-01-14 17:40:05,541 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793205289_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:05,541 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793205289_0001_00_000001 completed successfully
2016-01-14 17:40:05,546 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793205289_0001_1, queueSize=0
2016-01-14 17:40:05,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:05,546 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205289_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793205289_0001_1, startTime=1452793205452, finishTime=1452793205545, timeTaken=93, status=SUCCEEDED, diagnostics=, counters=Counters: 21, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793205516, Process_Duration=3, Process_End_Time=1452793205519, Read_Duration=1, Tuples_Read=10, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 17:40:05,546 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793205289_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:05,546 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793205289_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:05,547 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205289_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:05,547 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793205289_0001_00_000001
2016-01-14 17:40:05 Completed Dag: dag_1452793205289_0001_1
2016-01-14 17:40:05,547 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793205289_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:05,547 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793205289_0001_1_post
2016-01-14 17:40:05,547 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:05,547 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:05,554 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...m/testFilterAll/filterall, applicationId=application_1452793205289_0001
2016-01-14 17:40:05,557 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:05,557 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:05,557 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:05,557 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:05,557 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:05,560 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:05,560 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:05,561 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:05,561 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:05,561 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:05,561 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:05,562 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:05,562 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:05,568 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A401926396C546949E3870D71D4DAC94/E0D2DB1B83154315B01E547A86391CF2/.tez/application_1452793205289_0001
2016-01-14 17:40:05,569 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.300
2016-01-14 17:40:05,609 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilterAll/filterall
2016-01-14 17:40:05,638 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:05,638 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:05,684 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 17:40:05,703 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.035
2016-01-14 17:40:05,704 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:05,704 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:05,704 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:05,730 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 17:40:05,741 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-01-14 17:40:05,799 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 17:40:05,800 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:05,800 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-01-14 17:40:05,800 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 17:40:05,801 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 17:40:05,801 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 17:40:05,802 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...own/simplerelativeunknown
2016-01-14 17:40:05,802 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:05,818 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 17:40:05,823 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:05,823 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:05,823 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:05,832 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D/.tez/application_1452793205823_0001 doesn't exist and is created
2016-01-14 17:40:05,845 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:05,845 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:05,846 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D/.tez/application_1452793205823_0001
2016-01-14 17:40:05,848 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793205823_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:05,859 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:05,860 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:05,862 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:05,862 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:05,862 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D/.tez/application_1452793205823_0001/localmode-log-dir
2016-01-14 17:40:05,862 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D/.tez/application_1452793205823_0001/localmode-log-dir/history.txt.appattempt_1452793205823_0001_000000, maxErrors=10
2016-01-14 17:40:05,862 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:05,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793205823_0001_000000, appSubmitTime=1452793205847, launchTime=1452793205847
2016-01-14 17:40:05,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:05,870 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:58551
2016-01-14 17:40:05,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:05,872 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:05,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:05,873 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:05,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793205823_0001_000000, startTime=1452793205875
2016-01-14 17:40:05,875 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:05,875 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:05,946 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:05,946 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:05,946 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1452793205823_0001, dagName=[0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D] test/(1/1) ...own/simplerelativeunknown
2016-01-14 17:40:05,963 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D] test/(1/1) ...own/simplerelativeunknown
2016-01-14 17:40:05,965 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793205823_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D/.tez/application_1452793205823_0001/localmode-log-dir/dag_1452793205823_0001_1.dot
2016-01-14 17:40:05,966 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D/.tez/application_1452793205823_0001/localmode-log-dir/dag_1452793205823_0001_1-tez-dag.pb.txt
2016-01-14 17:40:05,980 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793205823_0001_1
2016-01-14 17:40:05,980 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:05,980 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D] test/(1/1) ...own/simplerelativeunknown
2016-01-14 17:40:05 Running Dag: dag_1452793205823_0001_1
2016-01-14 17:40:05,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793205823_0001_1, submitTime=1452793205964
2016-01-14 17:40:05,990 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:05,991 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C69BAC9C632B402086BDA41588498C92
2016-01-14 17:40:05,991 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:05,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793205823_0001_1, initTime=1452793205990
2016-01-14 17:40:05,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:05,992 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793205823_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:05,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:DAG_STARTED]: dagID=dag_1452793205823_0001_1, startTime=1452793205993
2016-01-14 17:40:05,993 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:05,993 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:05,993 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793205823_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:05,994 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FB439583186D400DAE0BFA44AA3B8807 : {151B398A5A2F4F3B943CB7B312B04432={InputName=151B398A5A2F4F3B943CB7B312B04432}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:05,994 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 151B398A5A2F4F3B943CB7B312B04432, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:05,994 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807]
2016-01-14 17:40:05,994 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807]
2016-01-14 17:40:05,994 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807]
2016-01-14 17:40:05,994 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807]. Starting root input initializers: 1
2016-01-14 17:40:05,995 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:05,995 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 151B398A5A2F4F3B943CB7B312B04432 on vertex vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807]
2016-01-14 17:40:05,996 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1452793205823_0001, dagName=[0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D] test/(1/1) ...own/simplerelativeunknown
2016-01-14 17:40:05,998 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:05,999 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 151B398A5A2F4F3B943CB7B312B04432 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:05,995 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92]
2016-01-14 17:40:06,001 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:06,003 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:40:06,003 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92]
2016-01-14 17:40:06,003 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92]
2016-01-14 17:40:06,003 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92]
2016-01-14 17:40:06,003 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=8FBD5AEE9B21483196328C5E538576CD, vertexId=vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:06,003 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=8FBD5AEE9B21483196328C5E538576CD, vertexId=vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92]
2016-01-14 17:40:06,005 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=8FBD5AEE9B21483196328C5E538576CD, vertexName=C69BAC9C632B402086BDA41588498C92, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:06,005 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=8FBD5AEE9B21483196328C5E538576CD, vertexId=vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92]
2016-01-14 17:40:06,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C69BAC9C632B402086BDA41588498C92, vertexId=vertex_1452793205823_0001_1_01, initRequestedTime=1452793205995, initedTime=1452793206006, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:40:06,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:06,003 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:06,006 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:40:06,010 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 17:40:06,010 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 151B398A5A2F4F3B943CB7B312B04432 on vertex vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807]
2016-01-14 17:40:06,011 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:06,011 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {151B398A5A2F4F3B943CB7B312B04432=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:06,012 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807] parallelism set to 1
2016-01-14 17:40:06,012 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:06,012 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793205823_0001_1_00, eventCount=1
2016-01-14 17:40:06,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:06,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FB439583186D400DAE0BFA44AA3B8807, vertexId=vertex_1452793205823_0001_1_00, initRequestedTime=1452793205994, initedTime=1452793206012, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:06,013 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807] with distanceFromRoot: 0
2016-01-14 17:40:06,014 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:06,014 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FB439583186D400DAE0BFA44AA3B8807
2016-01-14 17:40:06,014 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807]
2016-01-14 17:40:06,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793205823_0001_1_00, startRequestedTime=1452793206014, startedTime=1452793206014
2016-01-14 17:40:06,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:06,016 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:06,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:TASK_STARTED]: vertexName=FB439583186D400DAE0BFA44AA3B8807, taskId=task_1452793205823_0001_1_00_000000, scheduledTime=1452793206016, launchTime=1452793206016
2016-01-14 17:40:06,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:06,016 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793205823_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:06,017 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793205823_0001_1_00 for vertex: vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92] numStartedSources: 1 numSources: 1
2016-01-14 17:40:06,017 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92] with distanceFromRoot: 1
2016-01-14 17:40:06,018 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807]
2016-01-14 17:40:06,018 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452793205823_0001, with dag id: dag_1452793205823_0001_1
2016-01-14 17:40:06,019 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FB439583186D400DAE0BFA44AA3B8807 in vertex: C69BAC9C632B402086BDA41588498C92
2016-01-14 17:40:06,019 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C69BAC9C632B402086BDA41588498C92 with 1 source tasks and 1 pending tasks
2016-01-14 17:40:06,019 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793205823_0001_1_01, startRequestedTime=1452793206018, startedTime=1452793206018
2016-01-14 17:40:06,019 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:06,020 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:06,020 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D] test/(1/1) ...own/simplerelativeunknown, VertexName: FB439583186D400DAE0BFA44AA3B8807, VertexParallelism: 1, TaskAttemptID:attempt_1452793205823_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=151B398A5A2F4F3B943CB7B312B04432, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C69BAC9C632B402086BDA41588498C92, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:06,021 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793205823_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:06,021 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:06,023 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:06,023 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:06,024 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:06,024 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793205823_0001_00_000001
2016-01-14 17:40:06,024 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:06,026 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793205823_0001_00_000001
2016-01-14 17:40:06,026 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:06,028 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:06,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:06,028 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793205823_0001_00_000001, launchTime=1452793206028
2016-01-14 17:40:06,030 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793205823_0001_00_000001
2016-01-14 17:40:06,030 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793205823_0001_1_00_000000_0] to container: [container_1452793205823_0001_00_000001]
2016-01-14 17:40:06,030 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:06,031 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793205823_0001_1_00_000000_0] started. Is using containerId: [container_1452793205823_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:06,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FB439583186D400DAE0BFA44AA3B8807, taskAttemptId=attempt_1452793205823_0001_1_00_000000_0, startTime=1452793206031, containerId=container_1452793205823_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793205823_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:06,031 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793205823_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:06,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:06,031 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793205823_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:06,031 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793205823_0001_00_000001 given task: attempt_1452793205823_0001_1_00_000000_0
2016-01-14 17:40:06,031 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793205823_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793205823_0001_1_00_000000_0
2016-01-14 17:40:06 Starting to run new task attempt: attempt_1452793205823_0001_1_00_000000_0
2016-01-14 17:40:06,032 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793205823_0001_1_00_000000_0
2016-01-14 17:40:06,032 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,033 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:06,033 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:06,033 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:06,033 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:06,033 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:06,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D] test/(1/1) ...own/simplerelativeunknown, VertexName: FB439583186D400DAE0BFA44AA3B8807, VertexParallelism: 1, TaskAttemptID:attempt_1452793205823_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=151B398A5A2F4F3B943CB7B312B04432, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C69BAC9C632B402086BDA41588498C92, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:06,034 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:06,034 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793205823_0001_1_00_000000_0
2016-01-14 17:40:06,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:06,035 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=151B398A5A2F4F3B943CB7B312B04432, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:06,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C69BAC9C632B402086BDA41588498C92, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:06,036 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,037 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:06,041 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 151B398A5A2F4F3B943CB7B312B04432
2016-01-14 17:40:06,041 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 151B398A5A2F4F3B943CB7B312B04432
2016-01-14 17:40:06,040 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793205823_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:06,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:06,041 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:06,042 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C69BAC9C632B402086BDA41588498C92
2016-01-14 17:40:06,044 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:06,044 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C69BAC9C632B402086BDA41588498C92
2016-01-14 17:40:06,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C69BAC9C632B402086BDA41588498C92
2016-01-14 17:40:06,048 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FB439583186D400DAE0BFA44AA3B8807, ordinal: 0
2016-01-14 17:40:06,048 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:06,048 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FB439583186D400DAE0BFA44AA3B8807, mem on start (mb), free: 422, total: 542, max: 672
2016-01-14 17:40:06,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:06,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:06,049 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 151B398A5A2F4F3B943CB7B312B04432
2016-01-14 17:40:06,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 151B398A5A2F4F3B943CB7B312B04432
2016-01-14 17:40:06,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:06,050 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:06,051 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:06,051 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:06,051 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:06,051 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:06,051 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:06,051 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:40:06,052 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:40:06,052 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 151B398A5A2F4F3B943CB7B312B04432, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:06,052 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C69BAC9C632B402086BDA41588498C92, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:06,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:06,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 151B398A5A2F4F3B943CB7B312B04432 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:06,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:06,053 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:06,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 151B398A5A2F4F3B943CB7B312B04432
2016-01-14 17:40:06,053 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:06,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 151B398A5A2F4F3B943CB7B312B04432
2016-01-14 17:40:06,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:06,054 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793205823_0001_1_00_000000_0
2016-01-14 17:40:06,054 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:06,055 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@354906c9
2016-01-14 17:40:06,057 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@354906c9
2016-01-14 17:40:06,057 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:06,057 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:06,058 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:06,059 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 151B398A5A2F4F3B943CB7B312B04432
2016-01-14 17:40:06,059 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:-1]], id: D5E1049434C648939059687406F5A0E5
2016-01-14 17:40:06,059 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:-1]] D5E1049434C648939059687406F5A0E5
2016-01-14 17:40:06,060 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2bab35cc; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7558fa79; comparator=cascading.tuple.hadoop.util.TupleComparator@24a45a1f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:06,060 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:06,060 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:06,061 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:06,073 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:06,073 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:06,073 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:06,074 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:06,074 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:06,075 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextDelimited[[UNKNOWN]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:06,075 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:06,075 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FB439583186D400DAE0BFA44AA3B8807, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:06,078 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793205823_0001_1_00_000000_0
2016-01-14 17:40:06,078 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FB439583186D400DAE0BFA44AA3B8807, mem on close (mb), free: 320, total: 542, max: 672
2016-01-14 17:40:06,078 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:06,078 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:06,078 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 17:40:06,079 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 17:40:06,080 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:06 Completed running task attempt: attempt_1452793205823_0001_1_00_000000_0
2016-01-14 17:40:06,083 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793205823_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:06,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793205823_0001_1_00, eventCount=1
2016-01-14 17:40:06,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793206075, Process_Duration=3, Process_End_Time=1452793206078, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 17:40:06,084 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:06,085 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,085 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FB439583186D400DAE0BFA44AA3B8807, taskAttemptId=attempt_1452793205823_0001_1_00_000000_0, startTime=1452793206031, finishTime=1452793206084, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206075, Process_Duration=3, Process_End_Time=1452793206078, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:06,085 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793205823_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:06,085 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:06,085 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793205823_0001_1_00_000000_0
2016-01-14 17:40:06,089 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:TASK_FINISHED]: vertexName=FB439583186D400DAE0BFA44AA3B8807, taskId=task_1452793205823_0001_1_00_000000, startTime=1452793206031, finishTime=1452793206089, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1452793205823_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206075, Process_Duration=3, Process_End_Time=1452793206078, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:06,090 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793205823_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:06,090 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807] : 1
2016-01-14 17:40:06,090 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807]
2016-01-14 17:40:06,090 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:06,091 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:06,092 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:VERTEX_FINISHED]: vertexName=FB439583186D400DAE0BFA44AA3B8807, vertexId=vertex_1452793205823_0001_1_00, initRequestedTime=1452793205994, initedTime=1452793206012, startRequestedTime=1452793206014, startedTime=1452793206014, finishTime=1452793206090, timeTaken=76, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=62, FILE_BYTES_WRITTEN=127, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206075, Process_Duration=3, Process_End_Time=1452793206078, Read_Duration=3, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452793206031, firstTasksToStart=[ task_1452793205823_0001_1_00_000000 ], lastTaskFinishTime=1452793206084, lastTasksToFinish=[ task_1452793205823_0001_1_00_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793205823_0001_1_00_000000 ], longestDurationTasks=[ task_1452793205823_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:06,093 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:06,093 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:06,093 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92] attempt: attempt_1452793205823_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:06,093 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C69BAC9C632B402086BDA41588498C92
2016-01-14 17:40:06,093 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92]
2016-01-14 17:40:06,094 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:06,095 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793205823_0001_00_000001, stoppedTime=1452793206095, exitStatus=0
2016-01-14 17:40:06,095 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:06,096 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:06,096 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:TASK_STARTED]: vertexName=C69BAC9C632B402086BDA41588498C92, taskId=task_1452793205823_0001_1_01_000000, scheduledTime=1452793206096, launchTime=1452793206096
2016-01-14 17:40:06,095 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:06,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:06,097 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793205823_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:06,098 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92]
2016-01-14 17:40:06,098 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D] test/(1/1) ...own/simplerelativeunknown, VertexName: C69BAC9C632B402086BDA41588498C92, VertexParallelism: 1, TaskAttemptID:attempt_1452793205823_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FB439583186D400DAE0BFA44AA3B8807, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8FBD5AEE9B21483196328C5E538576CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:06,098 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793205823_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:06,099 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:06,099 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793205823_0001_00_000002
2016-01-14 17:40:06,100 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:06,097 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793205823_0001_00_000001
2016-01-14 17:40:06,098 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793205823_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:40:06,102 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:06,101 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793205823_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:06,102 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793205823_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:06,102 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793205823_0001_00_000001, Exiting
2016-01-14 17:40:06,103 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793205823_0001_00_000001
2016-01-14 17:40:06,103 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793205823_0001_00_000001
2016-01-14 17:40:06,103 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793205823_0001_00_000002
2016-01-14 17:40:06,107 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:40:06,104 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793205823_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:06,108 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793205823_0001_00_000001 completed successfully
2016-01-14 17:40:06,108 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:06,110 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:06,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:06,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793205823_0001_00_000002, launchTime=1452793206110
2016-01-14 17:40:06,111 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793205823_0001_00_000002
2016-01-14 17:40:06,112 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793205823_0001_1_01_000000_0] to container: [container_1452793205823_0001_00_000002]
2016-01-14 17:40:06,112 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:06,113 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793205823_0001_1_01_000000_0] started. Is using containerId: [container_1452793205823_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:40:06,113 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C69BAC9C632B402086BDA41588498C92, taskAttemptId=attempt_1452793205823_0001_1_01_000000_0, startTime=1452793206112, containerId=container_1452793205823_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793205823_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:40:06,113 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793205823_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:06,113 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:06,113 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793205823_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:06,114 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793205823_0001_00_000002 given task: attempt_1452793205823_0001_1_01_000000_0
2016-01-14 17:40:06,114 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793205823_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793205823_0001_1_01_000000_0
2016-01-14 17:40:06 Starting to run new task attempt: attempt_1452793205823_0001_1_01_000000_0
2016-01-14 17:40:06,114 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793205823_0001_1_01_000000_0
2016-01-14 17:40:06,114 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,115 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:06,115 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:06,115 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:06,115 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:06,115 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:06,115 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D] test/(1/1) ...own/simplerelativeunknown, VertexName: C69BAC9C632B402086BDA41588498C92, VertexParallelism: 1, TaskAttemptID:attempt_1452793205823_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FB439583186D400DAE0BFA44AA3B8807, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=8FBD5AEE9B21483196328C5E538576CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:06,116 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:06,116 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793205823_0001_1_01_000000_0
2016-01-14 17:40:06,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:06,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FB439583186D400DAE0BFA44AA3B8807, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:06,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=8FBD5AEE9B21483196328C5E538576CD, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:06,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:06,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:06,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 8FBD5AEE9B21483196328C5E538576CD
2016-01-14 17:40:06,119 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:06,119 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 8FBD5AEE9B21483196328C5E538576CD
2016-01-14 17:40:06,119 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793205823_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:40:06,120 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FB439583186D400DAE0BFA44AA3B8807
2016-01-14 17:40:06,121 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:06,121 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:06,122 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:06,122 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FB439583186D400DAE0BFA44AA3B8807
2016-01-14 17:40:06,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FB439583186D400DAE0BFA44AA3B8807
2016-01-14 17:40:06,130 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:06,130 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 8FBD5AEE9B21483196328C5E538576CD
2016-01-14 17:40:06,131 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C69BAC9C632B402086BDA41588498C92, ordinal: 1
2016-01-14 17:40:06,131 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C69BAC9C632B402086BDA41588498C92, mem on start (mb), free: 314, total: 542, max: 672
2016-01-14 17:40:06,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:06,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:06,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:06,133 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:06,134 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:06,135 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:06,135 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:06,135 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:06,135 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:40:06,137 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 8FBD5AEE9B21483196328C5E538576CD, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:06,137 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FB439583186D400DAE0BFA44AA3B8807, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:40:06,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:06,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FB439583186D400DAE0BFA44AA3B8807 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:06,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FB439583186D400DAE0BFA44AA3B8807
2016-01-14 17:40:06,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:06,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:06,138 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:06,139 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:06,140 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FB439583186D400DAE0BFA44AA3B8807 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:06,140 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:40:06,140 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:06,140 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:06,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FB439583186D400DAE0BFA44AA3B8807
2016-01-14 17:40:06,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:06,144 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793205823_0001_1_01_000000_0
2016-01-14 17:40:06,144 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793205823_0001_1_00_000000_0_10049, runDuration: 39076, ]
2016-01-14 17:40:06,146 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793205823_0001_1_00_000000_0_10049] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452793206146, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:40:06,146 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FB439583186D400DAE0BFA44AA3B8807
2016-01-14 17:40:06,146 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:06,146 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FB439583186D400DAE0BFA44AA3B8807] #8 in 1ms
2016-01-14 17:40:06,146 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FB439583186D400DAE0BFA44AA3B8807] #8, status:false, isInterrupted:false
2016-01-14 17:40:06,148 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:06,148 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 17:40:06,148 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:06,148 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:06,148 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 17:40:06,148 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FB439583186D400DAE0BFA44AA3B8807
2016-01-14 17:40:06,149 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:06,149 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:-1]]
2016-01-14 17:40:06,151 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:-1]] streamed: true, id: D5E1049434C648939059687406F5A0E5
2016-01-14 17:40:06,151 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"], id: 8FBD5AEE9B21483196328C5E538576CD
2016-01-14 17:40:06,152 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown"]
2016-01-14 17:40:06,152 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:-1]] D5E1049434C648939059687406F5A0E5, for 1 inputs
2016-01-14 17:40:06,153 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C69BAC9C632B402086BDA41588498C92, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:06,154 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1b96bc6b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@452c22f3
2016-01-14 17:40:06,156 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:06,156 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:06,156 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793205823_0001_1_01_000000_0
2016-01-14 17:40:06,157 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793205823_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:40:06,158 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793205823_0001_1_01_000000_0
2016-01-14 17:40:06,158 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C69BAC9C632B402086BDA41588498C92, mem on close (mb), free: 312, total: 542, max: 672
2016-01-14 17:40:06,159 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FB439583186D400DAE0BFA44AA3B8807
2016-01-14 17:40:06,159 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:06,160 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:06,160 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:06,160 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:06,161 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:06 Completed running task attempt: attempt_1452793205823_0001_1_01_000000_0
2016-01-14 17:40:06,163 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793205823_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:06,164 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C69BAC9C632B402086BDA41588498C92, taskAttemptId=attempt_1452793205823_0001_1_01_000000_0, startTime=1452793206112, finishTime=1452793206164, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206153, Process_Duration=5, Process_End_Time=1452793206158, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 17:40:06,164 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793205823_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:06,165 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:06,165 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793205823_0001_1_01_000000_0
2016-01-14 17:40:06,165 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:06,165 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:06,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:TASK_FINISHED]: vertexName=C69BAC9C632B402086BDA41588498C92, taskId=task_1452793205823_0001_1_01_000000, startTime=1452793206112, finishTime=1452793206165, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1452793205823_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206153, Process_Duration=5, Process_End_Time=1452793206158, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 17:40:06,166 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793205823_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:06,166 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92] : 1
2016-01-14 17:40:06,166 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92]
2016-01-14 17:40:06,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793206153, Process_Duration=5, Process_End_Time=1452793206158, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-01-14 17:40:06,167 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:06,167 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,167 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793205823_0001_00_000002
2016-01-14 17:40:06,167 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:06,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:VERTEX_FINISHED]: vertexName=C69BAC9C632B402086BDA41588498C92, vertexId=vertex_1452793205823_0001_1_01, initRequestedTime=1452793205995, initedTime=1452793206006, startRequestedTime=1452793206018, startedTime=1452793206018, finishTime=1452793206166, timeTaken=148, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=127, FILE_BYTES_WRITTEN=32, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=5, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=568852480, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206153, Process_Duration=5, Process_End_Time=1452793206158, Read_Duration=1, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1452793206112, firstTasksToStart=[ task_1452793205823_0001_1_01_000000 ], lastTaskFinishTime=1452793206164, lastTasksToFinish=[ task_1452793205823_0001_1_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793205823_0001_1_01_000000 ], longestDurationTasks=[ task_1452793205823_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:06,168 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:06,168 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:06,168 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793205823_0001_00_000002, stoppedTime=1452793206168, exitStatus=0
2016-01-14 17:40:06,169 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:06,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:06,169 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:06,169 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793205823_0001_1
2016-01-14 17:40:06,169 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793205823_0001_1
2016-01-14 17:40:06,169 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 8FBD5AEE9B21483196328C5E538576CD for vertex: vertex_1452793205823_0001_1_01 [C69BAC9C632B402086BDA41588498C92]
2016-01-14 17:40:06,176 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793205823_0001_1_00 [FB439583186D400DAE0BFA44AA3B8807]
2016-01-14 17:40:06,178 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793205823_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:40:06,178 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793205823_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:06,178 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793205823_0001_00_000002, Exiting
2016-01-14 17:40:06,178 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793205823_0001_00_000002
2016-01-14 17:40:06,178 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793205823_0001_00_000002
2016-01-14 17:40:06,179 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793205823_0001_1, queueSize=0
2016-01-14 17:40:06,179 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793205823_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:06,179 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:06,179 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793205823_0001_00_000002 completed successfully
2016-01-14 17:40:06,179 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793205823_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793205823_0001_1, startTime=1452793205993, finishTime=1452793206176, timeTaken=183, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=189, FILE_BYTES_WRITTEN=159, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=4, REDUCE_INPUT_RECORDS=5, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1137704960, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=95, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=95, SHUFFLE_BYTES_DECOMPRESSED=91, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=95, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905586412228, Process_Duration=8, Process_End_Time=2905586412236, Read_Duration=4, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, Tuples_Written=5
2016-01-14 17:40:06,179 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793205823_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:06,179 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793205823_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:06,180 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793205823_0001_00_000002
2016-01-14 17:40:06,180 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793205823_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:06 Completed Dag: dag_1452793205823_0001_1
2016-01-14 17:40:06,181 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793205823_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:06,181 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793205823_0001_1_post
2016-01-14 17:40:06,181 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,181 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:06,188 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...own/simplerelativeunknown, applicationId=application_1452793205823_0001
2016-01-14 17:40:06,190 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:06,190 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:06,190 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:06,190 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:06,190 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:06,191 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:06,191 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:06,191 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:06,192 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:06,192 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:06,192 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:06,192 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:06,192 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:06,197 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0D49800E615D42E69CD9892F714C688B/EA4CC57CE1EB4BB3AB6A3F2C80A0076D/.tez/application_1452793205823_0001
2016-01-14 17:40:06,197 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.398
2016-01-14 17:40:06,249 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimpleRelativeUnknown/simplerelativeunknown
2016-01-14 17:40:06,277 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:06,277 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:06,323 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 17:40:06,340 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-01-14 17:40:06,340 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 17:40:06,341 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 3
2016-01-14 17:40:06,341 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:06,359 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt
2016-01-14 17:40:06,366 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/upper.txt
2016-01-14 17:40:06,373 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-01-14 17:40:06,473 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting
2016-01-14 17:40:06,473 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 17:40:06,474 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:06,474 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-01-14 17:40:06,474 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] parallel execution of steps is enabled: false
2016-01-14 17:40:06,474 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] executing total steps: 1
2016-01-14 17:40:06,475 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] allocating management threads: 1
2016-01-14 17:40:06,475 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] starting step: (1/1) ...estCoGroup/complexcogroup
2016-01-14 17:40:06,476 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [lower*upper] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:06,489 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] tez session mode enabled: true
2016-01-14 17:40:06,494 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:06,494 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:06,494 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:06,503 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7/.tez/application_1452793206494_0001 doesn't exist and is created
2016-01-14 17:40:06,516 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:06,516 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:06,517 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7/.tez/application_1452793206494_0001
2016-01-14 17:40:06,519 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793206494_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:06,530 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:06,531 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:06,537 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:06,537 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:06,538 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7/.tez/application_1452793206494_0001/localmode-log-dir
2016-01-14 17:40:06,538 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7/.tez/application_1452793206494_0001/localmode-log-dir/history.txt.appattempt_1452793206494_0001_000000, maxErrors=10
2016-01-14 17:40:06,538 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:06,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793206494_0001_000000, appSubmitTime=1452793206518, launchTime=1452793206518
2016-01-14 17:40:06,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:06,543 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:06,544 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:06,546 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:55848
2016-01-14 17:40:06,548 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:06,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:06,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793206494_0001_000000, startTime=1452793206552
2016-01-14 17:40:06,552 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:06,552 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:06,616 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:06,617 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:06,617 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1452793206494_0001, dagName=[ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 17:40:06,637 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:40:06,637 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:40:06,637 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 17:40:06,639 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793206494_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7/.tez/application_1452793206494_0001/localmode-log-dir/dag_1452793206494_0001_1.dot
2016-01-14 17:40:06,640 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7/.tez/application_1452793206494_0001/localmode-log-dir/dag_1452793206494_0001_1-tez-dag.pb.txt
2016-01-14 17:40:06,655 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793206494_0001_1
2016-01-14 17:40:06,655 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,655 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 17:40:06 Running Dag: dag_1452793206494_0001_1
2016-01-14 17:40:06,666 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:06,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793206494_0001_1, submitTime=1452793206637
2016-01-14 17:40:06,668 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E4EDCD4FBCC146EA852BA06389FEDDD9
2016-01-14 17:40:06,668 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:06,669 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:06,669 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793206494_0001_1, initTime=1452793206667
2016-01-14 17:40:06,669 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793206494_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:06,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:DAG_STARTED]: dagID=dag_1452793206494_0001_1, startTime=1452793206669
2016-01-14 17:40:06,670 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:06,670 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793206494_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:06,670 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:06,670 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BBDB0FC50E974B45A8326230725B9D80 : {547263CEE6544DB19467F97E805CD2DE={InputName=547263CEE6544DB19467F97E805CD2DE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:06,670 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 547263CEE6544DB19467F97E805CD2DE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:06,670 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80]
2016-01-14 17:40:06,671 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80]
2016-01-14 17:40:06,671 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80]
2016-01-14 17:40:06,671 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80]. Starting root input initializers: 1
2016-01-14 17:40:06,672 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:06,672 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 547263CEE6544DB19467F97E805CD2DE on vertex vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80]
2016-01-14 17:40:06,672 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D21E11A56FE24E85B47B1C9EA4F9F8B0 : {59ACEE33A1304F6AB27FBAD6AAD32704={InputName=59ACEE33A1304F6AB27FBAD6AAD32704}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:06,672 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 59ACEE33A1304F6AB27FBAD6AAD32704, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:06,672 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0]
2016-01-14 17:40:06,673 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0]
2016-01-14 17:40:06,673 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0]
2016-01-14 17:40:06,673 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0]. Starting root input initializers: 1
2016-01-14 17:40:06,674 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:06,674 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:06,674 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9]
2016-01-14 17:40:06,674 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 59ACEE33A1304F6AB27FBAD6AAD32704 on vertex vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0]
2016-01-14 17:40:06,675 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 547263CEE6544DB19467F97E805CD2DE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:06,676 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:06,676 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1452793206494_0001, dagName=[ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7] lower*upper/(1/1) ...estCoGroup/complexcogroup
2016-01-14 17:40:06,677 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:06,677 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 59ACEE33A1304F6AB27FBAD6AAD32704 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:06,677 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:06,678 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:06,678 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:40:06,678 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9]
2016-01-14 17:40:06,678 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9]
2016-01-14 17:40:06,678 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9]
2016-01-14 17:40:06,678 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C1CD0360AD054762BABFD661434A8D27, vertexId=vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:06,679 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C1CD0360AD054762BABFD661434A8D27, vertexId=vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9]
2016-01-14 17:40:06,680 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 17:40:06,680 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 59ACEE33A1304F6AB27FBAD6AAD32704 on vertex vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0]
2016-01-14 17:40:06,681 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C1CD0360AD054762BABFD661434A8D27, vertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:06,681 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C1CD0360AD054762BABFD661434A8D27, vertexId=vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9]
2016-01-14 17:40:06,682 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:06,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, vertexId=vertex_1452793206494_0001_1_02, initRequestedTime=1452793206674, initedTime=1452793206682, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:40:06,683 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:40:06,683 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:06,683 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {59ACEE33A1304F6AB27FBAD6AAD32704=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:06,684 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0] parallelism set to 1
2016-01-14 17:40:06,684 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:06,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793206494_0001_1_01, eventCount=1
2016-01-14 17:40:06,685 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:06,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:06,686 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D21E11A56FE24E85B47B1C9EA4F9F8B0, vertexId=vertex_1452793206494_0001_1_01, initRequestedTime=1452793206672, initedTime=1452793206685, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:06,686 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0] with distanceFromRoot: 0
2016-01-14 17:40:06,686 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:06,686 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D21E11A56FE24E85B47B1C9EA4F9F8B0
2016-01-14 17:40:06,687 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0]
2016-01-14 17:40:06,687 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 522
2016-01-14 17:40:06,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:06,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793206494_0001_1_01, startRequestedTime=1452793206686, startedTime=1452793206686
2016-01-14 17:40:06,688 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:06,687 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 547263CEE6544DB19467F97E805CD2DE on vertex vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80]
2016-01-14 17:40:06,688 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:06,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_STARTED]: vertexName=D21E11A56FE24E85B47B1C9EA4F9F8B0, taskId=task_1452793206494_0001_1_01_000000, scheduledTime=1452793206688, launchTime=1452793206688
2016-01-14 17:40:06,688 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793206494_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:06,688 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793206494_0001_1_01 for vertex: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9] numStartedSources: 1 numSources: 2
2016-01-14 17:40:06,689 INFO impl.VertexImpl (VertexImpl.java:transition(3160)) - Cannot start vertex: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9] numStartedSources: 1 numSources: 2
2016-01-14 17:40:06,689 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0]
2016-01-14 17:40:06,689 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:06,689 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {547263CEE6544DB19467F97E805CD2DE=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:06,689 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80] parallelism set to 1
2016-01-14 17:40:06,689 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:06,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793206494_0001_1_00, eventCount=1
2016-01-14 17:40:06,690 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:06,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BBDB0FC50E974B45A8326230725B9D80, vertexId=vertex_1452793206494_0001_1_00, initRequestedTime=1452793206670, initedTime=1452793206690, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:06,691 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80] with distanceFromRoot: 0
2016-01-14 17:40:06,691 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:06,691 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: D21E11A56FE24E85B47B1C9EA4F9F8B0, VertexParallelism: 1, TaskAttemptID:attempt_1452793206494_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=59ACEE33A1304F6AB27FBAD6AAD32704, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:06,691 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793206494_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:06,691 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BBDB0FC50E974B45A8326230725B9D80
2016-01-14 17:40:06,691 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80]
2016-01-14 17:40:06,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:06,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793206494_0001_1_00, startRequestedTime=1452793206691, startedTime=1452793206691
2016-01-14 17:40:06,692 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:06,692 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:06,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:06,693 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_STARTED]: vertexName=BBDB0FC50E974B45A8326230725B9D80, taskId=task_1452793206494_0001_1_00_000000, scheduledTime=1452793206692, launchTime=1452793206692
2016-01-14 17:40:06,693 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793206494_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:06,693 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793206494_0001_1_00 for vertex: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9] numStartedSources: 2 numSources: 2
2016-01-14 17:40:06,693 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9] with distanceFromRoot: 1
2016-01-14 17:40:06,693 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80]
2016-01-14 17:40:06,693 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D21E11A56FE24E85B47B1C9EA4F9F8B0 in vertex: E4EDCD4FBCC146EA852BA06389FEDDD9
2016-01-14 17:40:06,694 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BBDB0FC50E974B45A8326230725B9D80 in vertex: E4EDCD4FBCC146EA852BA06389FEDDD9
2016-01-14 17:40:06,694 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E4EDCD4FBCC146EA852BA06389FEDDD9 with 2 source tasks and 1 pending tasks
2016-01-14 17:40:06,694 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:06,694 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793206494_0001_1_02, startRequestedTime=1452793206693, startedTime=1452793206693
2016-01-14 17:40:06,694 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:06,695 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: BBDB0FC50E974B45A8326230725B9D80, VertexParallelism: 1, TaskAttemptID:attempt_1452793206494_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=547263CEE6544DB19467F97E805CD2DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:06,695 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793206494_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:06,695 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:06,696 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:06,696 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:06,696 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:06,697 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793206494_0001_00_000001
2016-01-14 17:40:06,697 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:06,698 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793206494_0001_00_000002
2016-01-14 17:40:06,698 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:06,699 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793206494_0001_00_000001
2016-01-14 17:40:06,699 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:06,703 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793206494_0001_00_000002
2016-01-14 17:40:06,704 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:06,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:06,705 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:06,705 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:06,705 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:06,705 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793206494_0001_00_000001, launchTime=1452793206704
2016-01-14 17:40:06,705 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793206494_0001_00_000001
2016-01-14 17:40:06,706 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793206494_0001_00_000002, launchTime=1452793206704
2016-01-14 17:40:06,705 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] submitted tez dag to app master: application_1452793206494_0001, with dag id: dag_1452793206494_0001_1
2016-01-14 17:40:06,706 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793206494_0001_1_01_000000_0] to container: [container_1452793206494_0001_00_000001]
2016-01-14 17:40:06,706 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:06,709 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793206494_0001_00_000002
2016-01-14 17:40:06,709 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793206494_0001_00_000001 given task: attempt_1452793206494_0001_1_01_000000_0
2016-01-14 17:40:06,709 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793206494_0001_1_01_000000_0] started. Is using containerId: [container_1452793206494_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:06,710 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:06,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D21E11A56FE24E85B47B1C9EA4F9F8B0, taskAttemptId=attempt_1452793206494_0001_1_01_000000_0, startTime=1452793206709, containerId=container_1452793206494_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793206494_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:06,710 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793206494_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:06,710 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793206494_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793206494_0001_1_01_000000_0
2016-01-14 17:40:06 Starting to run new task attempt: attempt_1452793206494_0001_1_01_000000_0
2016-01-14 17:40:06,711 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793206494_0001_1_00_000000_0] to container: [container_1452793206494_0001_00_000002]
2016-01-14 17:40:06,712 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:06,712 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793206494_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:06,713 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793206494_0001_1_00_000000_0] started. Is using containerId: [container_1452793206494_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:40:06,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BBDB0FC50E974B45A8326230725B9D80, taskAttemptId=attempt_1452793206494_0001_1_00_000000_0, startTime=1452793206713, containerId=container_1452793206494_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793206494_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:40:06,713 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793206494_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:06,714 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793206494_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:06,711 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793206494_0001_1_01_000000_0
2016-01-14 17:40:06,714 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,714 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793206494_0001_00_000002 given task: attempt_1452793206494_0001_1_00_000000_0
2016-01-14 17:40:06,714 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:06,714 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:06,714 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:06,714 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:06,714 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:06,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: D21E11A56FE24E85B47B1C9EA4F9F8B0, VertexParallelism: 1, TaskAttemptID:attempt_1452793206494_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=59ACEE33A1304F6AB27FBAD6AAD32704, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:06,715 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:06,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:06,716 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793206494_0001_1_01_000000_0
2016-01-14 17:40:06,716 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793206494_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793206494_0001_1_00_000000_0
2016-01-14 17:40:06,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:06,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06 Starting to run new task attempt: attempt_1452793206494_0001_1_00_000000_0
2016-01-14 17:40:06,717 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793206494_0001_1_00_000000_0
2016-01-14 17:40:06,717 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,717 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:06,717 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:06,717 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:06,717 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:06,717 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:06,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=59ACEE33A1304F6AB27FBAD6AAD32704, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:06,719 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793206494_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:40:06,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:06,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 59ACEE33A1304F6AB27FBAD6AAD32704
2016-01-14 17:40:06,723 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 59ACEE33A1304F6AB27FBAD6AAD32704
2016-01-14 17:40:06,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:06,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: BBDB0FC50E974B45A8326230725B9D80, VertexParallelism: 1, TaskAttemptID:attempt_1452793206494_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=547263CEE6544DB19467F97E805CD2DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:06,727 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:06,727 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:06,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E4EDCD4FBCC146EA852BA06389FEDDD9
2016-01-14 17:40:06,729 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:06,730 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793206494_0001_1_00_000000_0
2016-01-14 17:40:06,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:06,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,726 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:06,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=547263CEE6544DB19467F97E805CD2DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:06,733 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:06,740 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E4EDCD4FBCC146EA852BA06389FEDDD9
2016-01-14 17:40:06,733 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:06,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:06,733 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793206494_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:06,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,731 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 59ACEE33A1304F6AB27FBAD6AAD32704
2016-01-14 17:40:06,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 59ACEE33A1304F6AB27FBAD6AAD32704
2016-01-14 17:40:06,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E4EDCD4FBCC146EA852BA06389FEDDD9
2016-01-14 17:40:06,744 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:06,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:06,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E4EDCD4FBCC146EA852BA06389FEDDD9
2016-01-14 17:40:06,745 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E4EDCD4FBCC146EA852BA06389FEDDD9
2016-01-14 17:40:06,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 547263CEE6544DB19467F97E805CD2DE
2016-01-14 17:40:06,745 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:06,745 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 547263CEE6544DB19467F97E805CD2DE
2016-01-14 17:40:06,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E4EDCD4FBCC146EA852BA06389FEDDD9
2016-01-14 17:40:06,747 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D21E11A56FE24E85B47B1C9EA4F9F8B0, ordinal: 1
2016-01-14 17:40:06,747 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D21E11A56FE24E85B47B1C9EA4F9F8B0, mem on start (mb), free: 402, total: 543, max: 672
2016-01-14 17:40:06,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:06,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:06,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:06,750 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:06,751 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:06,752 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:06,752 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:06,752 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:06,752 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:06,753 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 59ACEE33A1304F6AB27FBAD6AAD32704, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:06,753 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E4EDCD4FBCC146EA852BA06389FEDDD9, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:06,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:06,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 59ACEE33A1304F6AB27FBAD6AAD32704 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:06,751 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:06,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 59ACEE33A1304F6AB27FBAD6AAD32704
2016-01-14 17:40:06,754 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:06,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 59ACEE33A1304F6AB27FBAD6AAD32704
2016-01-14 17:40:06,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:06,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:06,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:06,756 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793206494_0001_1_01_000000_0
2016-01-14 17:40:06,757 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BBDB0FC50E974B45A8326230725B9D80, ordinal: 0
2016-01-14 17:40:06,757 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BBDB0FC50E974B45A8326230725B9D80, mem on start (mb), free: 401, total: 543, max: 672
2016-01-14 17:40:06,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:06,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:06,754 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 547263CEE6544DB19467F97E805CD2DE
2016-01-14 17:40:06,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 547263CEE6544DB19467F97E805CD2DE
2016-01-14 17:40:06,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:06,759 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:06,759 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 17:40:06,760 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/upper.txt"] streamed: true, id: 59ACEE33A1304F6AB27FBAD6AAD32704
2016-01-14 17:40:06,760 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: 9E7FD0524A2C4DD39543D906E2384485
2016-01-14 17:40:06,760 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 9E7FD0524A2C4DD39543D906E2384485
2016-01-14 17:40:06,761 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:06,761 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1346e527; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@136e6d54; comparator=cascading.tuple.hadoop.util.TupleComparator@36cbafc3; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:06,763 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7680474f
2016-01-14 17:40:06,761 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:06,763 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:06,763 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:06,763 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:06,763 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:06,776 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:06,776 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:06,776 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:06,777 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E4EDCD4FBCC146EA852BA06389FEDDD9, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:06,777 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 547263CEE6544DB19467F97E805CD2DE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:06,777 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:06,777 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 547263CEE6544DB19467F97E805CD2DE
2016-01-14 17:40:06,777 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:06,777 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 547263CEE6544DB19467F97E805CD2DE
2016-01-14 17:40:06,778 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 547263CEE6544DB19467F97E805CD2DE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:06,777 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7680474f
2016-01-14 17:40:06,778 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:06,779 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:06,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:06,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:06,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:06,782 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793206494_0001_1_00_000000_0
2016-01-14 17:40:06,783 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:06,785 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@73bd4585
2016-01-14 17:40:06,789 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:06,789 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:06,790 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"] streamed: true, id: 547263CEE6544DB19467F97E805CD2DE
2016-01-14 17:40:06,790 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]], id: 9E7FD0524A2C4DD39543D906E2384485
2016-01-14 17:40:06,790 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 9E7FD0524A2C4DD39543D906E2384485
2016-01-14 17:40:06,791 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@42aab93; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@d664208; comparator=cascading.tuple.hadoop.util.TupleComparator@34792217; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:06,789 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:06,792 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:06,792 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:06,793 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:06,792 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:06,793 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:06,793 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:06,791 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@73bd4585
2016-01-14 17:40:06,794 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:06,795 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:06,794 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/upper.txt"]
2016-01-14 17:40:06,796 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:06,796 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D21E11A56FE24E85B47B1C9EA4F9F8B0, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:06,806 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:06,806 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:06,806 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:06,807 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:06,807 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:06,808 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/lower.txt"]
2016-01-14 17:40:06,808 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:06,808 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BBDB0FC50E974B45A8326230725B9D80, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:06,812 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793206494_0001_1_01_000000_0
2016-01-14 17:40:06,812 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D21E11A56FE24E85B47B1C9EA4F9F8B0, mem on close (mb), free: 310, total: 543, max: 672
2016-01-14 17:40:06,813 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:06,813 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:06,813 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 17:40:06,813 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 17:40:06,813 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793206494_0001_1_00_000000_0
2016-01-14 17:40:06,814 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BBDB0FC50E974B45A8326230725B9D80, mem on close (mb), free: 310, total: 543, max: 672
2016-01-14 17:40:06,814 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:06,814 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:06,814 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 75; bufvoid = 104857600
2016-01-14 17:40:06,815 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214380(104857520); length = 17/6553600
2016-01-14 17:40:06,816 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:06,816 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:06 Completed running task attempt: attempt_1452793206494_0001_1_01_000000_0
2016-01-14 17:40:06 Completed running task attempt: attempt_1452793206494_0001_1_00_000000_0
2016-01-14 17:40:06,820 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793206494_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:06,821 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793206494_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:06,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793206494_0001_1_01, eventCount=1
2016-01-14 17:40:06,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793206809, Process_Duration=4, Process_End_Time=1452793206813, Read_Duration=4, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 17:40:06,823 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793206494_0001_1_00, eventCount=1
2016-01-14 17:40:06,823 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:06,824 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,824 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793206494_0001_00_000002
2016-01-14 17:40:06,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D21E11A56FE24E85B47B1C9EA4F9F8B0, taskAttemptId=attempt_1452793206494_0001_1_01_000000_0, startTime=1452793206709, finishTime=1452793206824, timeTaken=115, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206796, Process_Duration=16, Process_End_Time=1452793206812, Read_Duration=14, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:06,824 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793206494_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:06,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BBDB0FC50E974B45A8326230725B9D80, taskAttemptId=attempt_1452793206494_0001_1_00_000000_0, startTime=1452793206713, finishTime=1452793206824, timeTaken=111, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206809, Process_Duration=4, Process_End_Time=1452793206813, Read_Duration=4, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:06,825 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793206494_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:06,825 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793206494_0001_1_01_000000_0
2016-01-14 17:40:06,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_FINISHED]: vertexName=D21E11A56FE24E85B47B1C9EA4F9F8B0, taskId=task_1452793206494_0001_1_01_000000, startTime=1452793206709, finishTime=1452793206825, timeTaken=116, status=SUCCEEDED, successfulAttemptID=attempt_1452793206494_0001_1_01_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206796, Process_Duration=16, Process_End_Time=1452793206812, Read_Duration=14, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:06,826 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793206494_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:06,826 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793206494_0001_1_00_000000_0
2016-01-14 17:40:06,827 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_FINISHED]: vertexName=BBDB0FC50E974B45A8326230725B9D80, taskId=task_1452793206494_0001_1_00_000000, startTime=1452793206713, finishTime=1452793206826, timeTaken=113, status=SUCCEEDED, successfulAttemptID=attempt_1452793206494_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206809, Process_Duration=4, Process_End_Time=1452793206813, Read_Duration=4, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5
2016-01-14 17:40:06,827 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793206494_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:06,827 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0] : 1
2016-01-14 17:40:06,827 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0]
2016-01-14 17:40:06,828 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:06,828 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:06,829 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:06,830 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:06,830 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:06,831 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:06,831 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:06,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_FINISHED]: vertexName=D21E11A56FE24E85B47B1C9EA4F9F8B0, vertexId=vertex_1452793206494_0001_1_01, initRequestedTime=1452793206672, initedTime=1452793206685, startRequestedTime=1452793206686, startedTime=1452793206686, finishTime=1452793206827, timeTaken=141, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206796, Process_Duration=16, Process_End_Time=1452793206812, Read_Duration=14, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452793206709, firstTasksToStart=[ task_1452793206494_0001_1_01_000000 ], lastTaskFinishTime=1452793206824, lastTasksToFinish=[ task_1452793206494_0001_1_01_000000 ], minTaskDuration=115, maxTaskDuration=115, avgTaskDuration=115.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793206494_0001_1_01_000000 ], longestDurationTasks=[ task_1452793206494_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:06,832 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:06,832 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80] : 1
2016-01-14 17:40:06,833 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80]
2016-01-14 17:40:06,833 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:06,834 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_FINISHED]: vertexName=BBDB0FC50E974B45A8326230725B9D80, vertexId=vertex_1452793206494_0001_1_00, initRequestedTime=1452793206670, initedTime=1452793206690, startRequestedTime=1452793206691, startedTime=1452793206691, finishTime=1452793206833, timeTaken=142, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206809, Process_Duration=4, Process_End_Time=1452793206813, Read_Duration=4, Tuples_Read=5, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=5, vertexStats=firstTaskStartTime=1452793206713, firstTasksToStart=[ task_1452793206494_0001_1_00_000000 ], lastTaskFinishTime=1452793206824, lastTasksToFinish=[ task_1452793206494_0001_1_00_000000 ], minTaskDuration=111, maxTaskDuration=111, avgTaskDuration=111.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793206494_0001_1_00_000000 ], longestDurationTasks=[ task_1452793206494_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:06,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=124, FILE_BYTES_WRITTEN=254, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=5, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=5, OUTPUT_BYTES=75, OUTPUT_BYTES_WITH_OVERHEAD=91, OUTPUT_BYTES_PHYSICAL=95, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793206796, Process_Duration=16, Process_End_Time=1452793206812, Read_Duration=14, Tuples_Read=5, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=5]]
2016-01-14 17:40:06,835 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:06,835 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,835 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793206494_0001_00_000001
2016-01-14 17:40:06,835 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:06,836 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9] attempt: attempt_1452793206494_0001_1_01_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:06,836 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:06,836 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:06,837 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793206494_0001_00_000001, stoppedTime=1452793206836, exitStatus=0
2016-01-14 17:40:06,837 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:06,837 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:06,837 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:06,838 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793206494_0001_00_000002, stoppedTime=1452793206837, exitStatus=0
2016-01-14 17:40:06,838 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:06,838 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 17:40:06,838 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9] attempt: attempt_1452793206494_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:06,838 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E4EDCD4FBCC146EA852BA06389FEDDD9
2016-01-14 17:40:06,839 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9]
2016-01-14 17:40:06,839 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 17:40:06,839 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793206494_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:40:06,839 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793206494_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 17:40:06,839 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:06,839 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_STARTED]: vertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, taskId=task_1452793206494_0001_1_02_000000, scheduledTime=1452793206839, launchTime=1452793206839
2016-01-14 17:40:06,840 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793206494_0001_1_02_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:06,840 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9]
2016-01-14 17:40:06,841 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:06,841 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:06,841 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: E4EDCD4FBCC146EA852BA06389FEDDD9, VertexParallelism: 1, TaskAttemptID:attempt_1452793206494_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D21E11A56FE24E85B47B1C9EA4F9F8B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=BBDB0FC50E974B45A8326230725B9D80, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C1CD0360AD054762BABFD661434A8D27, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:06,841 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793206494_0001_1_02_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:06,842 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:06,842 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793206494_0001_00_000003
2016-01-14 17:40:06,842 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000003 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:06,844 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793206494_0001_00_000003
2016-01-14 17:40:06,844 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:06,845 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000003 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:06,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:06,845 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793206494_0001_00_000003, launchTime=1452793206845
2016-01-14 17:40:06,846 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793206494_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:06,846 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793206494_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:06,846 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793206494_0001_00_000001, Exiting
2016-01-14 17:40:06,846 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793206494_0001_00_000001
2016-01-14 17:40:06,846 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793206494_0001_00_000001
2016-01-14 17:40:06,846 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793206494_0001_00_000003
2016-01-14 17:40:06,850 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793206494_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:06,850 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793206494_0001_00_000001 completed successfully
2016-01-14 17:40:06,850 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793206494_0001_1_02_000000_0] to container: [container_1452793206494_0001_00_000003]
2016-01-14 17:40:06,851 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:06,851 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000003 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:06,852 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793206494_0001_00_000003 given task: attempt_1452793206494_0001_1_02_000000_0
2016-01-14 17:40:06,852 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793206494_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793206494_0001_1_02_000000_0
2016-01-14 17:40:06 Starting to run new task attempt: attempt_1452793206494_0001_1_02_000000_0
2016-01-14 17:40:06,852 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793206494_0001_1_02_000000_0
2016-01-14 17:40:06,852 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,853 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:06,853 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:06,853 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:06,853 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:06,853 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:06,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7] lower*upper/(1/1) ...estCoGroup/complexcogroup, VertexName: E4EDCD4FBCC146EA852BA06389FEDDD9, VertexParallelism: 1, TaskAttemptID:attempt_1452793206494_0001_1_02_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=2, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D21E11A56FE24E85B47B1C9EA4F9F8B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, {{ sourceVertexName=BBDB0FC50E974B45A8326230725B9D80, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C1CD0360AD054762BABFD661434A8D27, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:06,853 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=2, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:06,854 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793206494_0001_1_02_000000_0] started. Is using containerId: [container_1452793206494_0001_00_000003] on NM: [127.0.0.1:0]
2016-01-14 17:40:06,854 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793206494_0001_1_02_000000_0
2016-01-14 17:40:06,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:06,855 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793206494_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:40:06,855 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793206494_0001_00_000002: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:06,855 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:06,855 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793206494_0001_1_02_000000_0 sent events: (0-2)
2016-01-14 17:40:06,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, taskAttemptId=attempt_1452793206494_0001_1_02_000000_0, startTime=1452793206852, containerId=container_1452793206494_0001_00_000003, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793206494_0001_00_000003/teamcity, completedLogs=
2016-01-14 17:40:06,855 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793206494_0001_00_000002, Exiting
2016-01-14 17:40:06,856 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793206494_0001_00_000002
2016-01-14 17:40:06,856 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793206494_0001_00_000002
2016-01-14 17:40:06,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BBDB0FC50E974B45A8326230725B9D80, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:06,856 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793206494_0001_1_02_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:06,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:06,857 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793206494_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:06,856 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,857 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793206494_0001_00_000002 completed successfully
2016-01-14 17:40:06,857 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BBDB0FC50E974B45A8326230725B9D80
2016-01-14 17:40:06,861 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:06,857 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793206494_0001_1_02_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:06,861 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:06,861 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:06,862 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:06,862 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BBDB0FC50E974B45A8326230725B9D80
2016-01-14 17:40:06,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BBDB0FC50E974B45A8326230725B9D80
2016-01-14 17:40:06,863 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D21E11A56FE24E85B47B1C9EA4F9F8B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:06,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:06,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D21E11A56FE24E85B47B1C9EA4F9F8B0
2016-01-14 17:40:06,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C1CD0360AD054762BABFD661434A8D27, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:06,866 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:06,866 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:06,866 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D21E11A56FE24E85B47B1C9EA4F9F8B0
2016-01-14 17:40:06,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:06,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C1CD0360AD054762BABFD661434A8D27
2016-01-14 17:40:06,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D21E11A56FE24E85B47B1C9EA4F9F8B0
2016-01-14 17:40:06,867 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:06,868 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C1CD0360AD054762BABFD661434A8D27
2016-01-14 17:40:06,869 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E4EDCD4FBCC146EA852BA06389FEDDD9, ordinal: 2
2016-01-14 17:40:06,869 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4EDCD4FBCC146EA852BA06389FEDDD9, mem on start (mb), free: 302, total: 543, max: 672
2016-01-14 17:40:06,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:06,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 3 initializers to finish
2016-01-14 17:40:06,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:06,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:06,877 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:06,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C1CD0360AD054762BABFD661434A8D27
2016-01-14 17:40:06,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:06,877 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:06,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:06,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:06,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:06,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:06,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.045, finalReserveFractionUsed=0.345
2016-01-14 17:40:06,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 3, numScaledRequests: 25, TotalRequested: 1268357504, TotalRequestedScaled: 6.0881160192E8, TotalJVMHeap: 704643072, TotalAvailable: 461541212, TotalRequested/TotalJVMHeap:1.80
2016-01-14 17:40:06,879 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BBDB0FC50E974B45A8326230725B9D80, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 17:40:06,879 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D21E11A56FE24E85B47B1C9EA4F9F8B0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=230770606
2016-01-14 17:40:06,879 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C1CD0360AD054762BABFD661434A8D27, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:06,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:06,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D21E11A56FE24E85B47B1C9EA4F9F8B0 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:06,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D21E11A56FE24E85B47B1C9EA4F9F8B0
2016-01-14 17:40:06,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BBDB0FC50E974B45A8326230725B9D80
2016-01-14 17:40:06,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BBDB0FC50E974B45A8326230725B9D80 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:06,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 2
2016-01-14 17:40:06,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 2 IOs to start
2016-01-14 17:40:06,886 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:06,887 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:06,888 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D21E11A56FE24E85B47B1C9EA4F9F8B0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:06,888 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 17:40:06,888 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:06,889 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:06,889 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:06,889 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:06,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D21E11A56FE24E85B47B1C9EA4F9F8B0
2016-01-14 17:40:06,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:06,891 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BBDB0FC50E974B45A8326230725B9D80 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:06,891 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=230770606. Updated to: ShuffleMem=230770606, postMergeMem: 0
2016-01-14 17:40:06,891 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=230770606, maxSingleShuffleLimit=57692652, mergeThreshold=207693536, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:06,892 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:06,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BBDB0FC50E974B45A8326230725B9D80
2016-01-14 17:40:06,893 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:06,894 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793206494_0001_1_02_000000_0
2016-01-14 17:40:06,894 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793206494_0001_1_01_000000_0_10055, runDuration: 88941, ]
2016-01-14 17:40:06,895 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793206494_0001_1_00_000000_0_10057, runDuration: 75006, ]
2016-01-14 17:40:06,895 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793206494_0001_1_01_000000_0_10055] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452793206895, TimeTaken=1, Rate=0.09 MB/s
2016-01-14 17:40:06,896 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D21E11A56FE24E85B47B1C9EA4F9F8B0
2016-01-14 17:40:06,896 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:06,896 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D21E11A56FE24E85B47B1C9EA4F9F8B0] #9 in 2ms
2016-01-14 17:40:06,908 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D21E11A56FE24E85B47B1C9EA4F9F8B0] #9, status:false, isInterrupted:false
2016-01-14 17:40:06,909 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:06,909 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 17:40:06,909 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:06,909 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:06,909 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793206494_0001_1_00_000000_0_10057] to DISK_DIRECT, CompressedSize=95, DecompressedSize=91,EndTime=1452793206909, TimeTaken=1, Rate=0.09 MB/s
2016-01-14 17:40:06,910 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BBDB0FC50E974B45A8326230725B9D80
2016-01-14 17:40:06,910 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 17:40:06,910 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D21E11A56FE24E85B47B1C9EA4F9F8B0
2016-01-14 17:40:06,910 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:06,910 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:06,910 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]]
2016-01-14 17:40:06,911 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BBDB0FC50E974B45A8326230725B9D80] #10 in 4ms
2016-01-14 17:40:06,912 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] streamed: true, id: 9E7FD0524A2C4DD39543D906E2384485
2016-01-14 17:40:06,912 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"], id: C1CD0360AD054762BABFD661434A8D27
2016-01-14 17:40:06,913 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup"]
2016-01-14 17:40:06,913 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 9E7FD0524A2C4DD39543D906E2384485, for 2 inputs
2016-01-14 17:40:06,913 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: CoGroup(lower*upper)[by: lower:[{1}:0] upper:[{1}:0]] 9E7FD0524A2C4DD39543D906E2384485, for 2 inputs
2016-01-14 17:40:06,916 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BBDB0FC50E974B45A8326230725B9D80] #10, status:false, isInterrupted:false
2016-01-14 17:40:06,918 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:06,918 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 95 bytes from disk
2016-01-14 17:40:06,918 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:06,918 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:06,918 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 78 bytes
2016-01-14 17:40:06,919 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BBDB0FC50E974B45A8326230725B9D80
2016-01-14 17:40:06,919 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:06,918 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(89)) - attempting to load codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 17:40:06,919 INFO collect.SpillableTupleList (SpillableTupleList.java:getCodecClass(94)) - found codec: org.apache.hadoop.io.compress.GzipCodec
2016-01-14 17:40:06,920 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E4EDCD4FBCC146EA852BA06389FEDDD9, all 2 inputs ready in: 00:00:00.000
2016-01-14 17:40:06,920 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@377526; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@38ec11e6
2016-01-14 17:40:06,921 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@128b303c; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@700bbca6
2016-01-14 17:40:06,925 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:06,925 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:06,926 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793206494_0001_1_02_000000_0
2016-01-14 17:40:06,926 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793206494_0001_1_02_000000_0 given a go for committing the task output.
2016-01-14 17:40:06,927 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793206494_0001_1_02_000000_0
2016-01-14 17:40:06,927 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E4EDCD4FBCC146EA852BA06389FEDDD9, mem on close (mb), free: 513, total: 543, max: 672
2016-01-14 17:40:06,927 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D21E11A56FE24E85B47B1C9EA4F9F8B0
2016-01-14 17:40:06,927 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:06,928 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:06,928 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:06,928 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:06,928 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:06,928 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BBDB0FC50E974B45A8326230725B9D80
2016-01-14 17:40:06,929 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:06,929 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:06,929 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:06,930 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:06,930 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:06 Completed running task attempt: attempt_1452793206494_0001_1_02_000000_0
2016-01-14 17:40:06,931 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793206494_0001_1_02_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:06,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, taskAttemptId=attempt_1452793206494_0001_1_02_000000_0, startTime=1452793206852, finishTime=1452793206933, timeTaken=81, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=572, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=22, MERGE_PHASE_TIME=30, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206920, Process_Duration=7, Process_End_Time=1452793206927, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 17:40:06,933 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793206494_0001_1_02_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:06,933 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:06,933 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793206494_0001_1_02_000000_0
2016-01-14 17:40:06,934 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:06,934 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:TASK_FINISHED]: vertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, taskId=task_1452793206494_0001_1_02_000000, startTime=1452793206852, finishTime=1452793206933, timeTaken=81, status=SUCCEEDED, successfulAttemptID=attempt_1452793206494_0001_1_02_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=572, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=22, MERGE_PHASE_TIME=30, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206920, Process_Duration=7, Process_End_Time=1452793206927, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5
2016-01-14 17:40:06,934 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793206494_0001_1_02_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:06,934 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:06,935 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9] : 1
2016-01-14 17:40:06,935 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9]
2016-01-14 17:40:06,936 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:06,936 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:VERTEX_FINISHED]: vertexName=E4EDCD4FBCC146EA852BA06389FEDDD9, vertexId=vertex_1452793206494_0001_1_02, initRequestedTime=1452793206674, initedTime=1452793206682, startRequestedTime=1452793206693, startedTime=1452793206693, finishTime=1452793206935, timeTaken=242, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=572, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=22, MERGE_PHASE_TIME=30, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793206920, Process_Duration=7, Process_End_Time=1452793206927, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=5, vertexStats=firstTaskStartTime=1452793206852, firstTasksToStart=[ task_1452793206494_0001_1_02_000000 ], lastTaskFinishTime=1452793206933, lastTasksToFinish=[ task_1452793206494_0001_1_02_000000 ], minTaskDuration=81, maxTaskDuration=81, avgTaskDuration=81.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793206494_0001_1_02_000000 ], longestDurationTasks=[ task_1452793206494_0001_1_02_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:06,936 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:06,937 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000003 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:06,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=254, FILE_BYTES_WRITTEN=572, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, OUTPUT_RECORDS=5, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=22, MERGE_PHASE_TIME=30, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793206920, Process_Duration=7, Process_End_Time=1452793206927, Read_Duration=2, Tuples_Read=10, Tuples_Written=5, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=5]]
2016-01-14 17:40:06,937 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:06,937 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793206494_0001_00_000003, stoppedTime=1452793206937, exitStatus=0
2016-01-14 17:40:06,939 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:06,939 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,939 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000003 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:06,939 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9] completed., numCompletedVertices=3, numSuccessfulVertices=3, numFailedVertices=0, numKilledVertices=0, numVertices=3
2016-01-14 17:40:06,939 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793206494_0001_1
2016-01-14 17:40:06,940 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793206494_0001_00_000003
2016-01-14 17:40:06,940 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793206494_0001_00_000003 is valid, but no longer registered, and will be killed
2016-01-14 17:40:06,940 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793206494_0001_1
2016-01-14 17:40:06,940 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793206494_0001_1_00 [BBDB0FC50E974B45A8326230725B9D80]
2016-01-14 17:40:06,940 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793206494_0001_1_01 [D21E11A56FE24E85B47B1C9EA4F9F8B0]
2016-01-14 17:40:06,940 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C1CD0360AD054762BABFD661434A8D27 for vertex: vertex_1452793206494_0001_1_02 [E4EDCD4FBCC146EA852BA06389FEDDD9]
2016-01-14 17:40:06,940 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793206494_0001_00_000003: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:06,940 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793206494_0001_00_000003, Exiting
2016-01-14 17:40:06,941 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793206494_0001_00_000003
2016-01-14 17:40:06,941 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793206494_0001_00_000003
2016-01-14 17:40:06,941 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793206494_0001_00_000003 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:06,941 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793206494_0001_00_000003 completed successfully
2016-01-14 17:40:06,952 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793206494_0001_1, queueSize=0
2016-01-14 17:40:06,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:06,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793206494_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793206494_0001_1, startTime=1452793206669, finishTime=1452793206947, timeTaken=278, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=3, TOTAL_LAUNCHED_TASKS=3, RACK_LOCAL_TASKS=2, File System Counters, FILE_BYTES_READ=502, FILE_BYTES_WRITTEN=1080, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=8, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=2, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=2, GC_TIME_MILLIS=29, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1708654592, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=15, OUTPUT_BYTES=150, OUTPUT_BYTES_WITH_OVERHEAD=182, OUTPUT_BYTES_PHYSICAL=190, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=190, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=190, SHUFFLE_BYTES_DECOMPRESSED=182, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=190, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=22, MERGE_PHASE_TIME=30, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=4358379620525, Process_Duration=27, Process_End_Time=4358379620552, Read_Duration=20, Tuples_Read=20, Tuples_Written=15, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=5
2016-01-14 17:40:06,954 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793206494_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:06,954 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793206494_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:06,955 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793206494_0001_00_000003
2016-01-14 17:40:06,956 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793206494_0001_00_000003 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:06 Completed Dag: dag_1452793206494_0001_1
2016-01-14 17:40:06,957 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793206494_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:06,958 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793206494_0001_1_post
2016-01-14 17:40:06,958 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:06,958 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:06,962 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...estCoGroup/complexcogroup, applicationId=application_1452793206494_0001
2016-01-14 17:40:06,966 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:06,966 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:06,966 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:06,966 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:06,966 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:06,966 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:06,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:06,967 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:06,967 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:06,967 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:06,968 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:06,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:06,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:06,973 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/ACE2D843092443F587283AF79C036F26/EE900534204C4A51A3E55EAC29C3F4A7/.tez/application_1452793206494_0001
2016-01-14 17:40:06,974 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [lower*upper] completed in: 00:00.501
2016-01-14 17:40:07,055 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCoGroup/complexcogroup
2016-01-14 17:40:07,093 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:07,094 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:07,133 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 17:40:07,143 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:40:07,143 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:40:07,148 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-01-14 17:40:07,148 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 17:40:07,149 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 17:40:07,150 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:07,167 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 17:40:07,168 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace
2016-01-14 17:40:07,178 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 17:40:07,178 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:07,178 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-01-14 17:40:07,178 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 17:40:07,179 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 17:40:07,179 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 17:40:07,180 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...tform/testReplace/replace
2016-01-14 17:40:07,180 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:07,201 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 17:40:07,205 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:07,206 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:07,206 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:07,215 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33/.tez/application_1452793207206_0001 doesn't exist and is created
2016-01-14 17:40:07,229 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:07,230 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:07,230 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33/.tez/application_1452793207206_0001
2016-01-14 17:40:07,232 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793207206_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:07,243 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:07,244 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:07,246 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:07,246 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:07,246 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33/.tez/application_1452793207206_0001/localmode-log-dir
2016-01-14 17:40:07,247 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33/.tez/application_1452793207206_0001/localmode-log-dir/history.txt.appattempt_1452793207206_0001_000000, maxErrors=10
2016-01-14 17:40:07,247 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:07,247 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793207206_0001_000000, appSubmitTime=1452793207231, launchTime=1452793207232
2016-01-14 17:40:07,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:07,253 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:07,254 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:57323
2016-01-14 17:40:07,255 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:07,256 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:07,257 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:07,260 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793207206_0001_000000, startTime=1452793207260
2016-01-14 17:40:07,260 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:07,260 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:07,330 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:07,330 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:07,330 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1452793207206_0001, dagName=[C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33] test/(1/1) ...tform/testReplace/replace
2016-01-14 17:40:07,355 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33] test/(1/1) ...tform/testReplace/replace
2016-01-14 17:40:07,357 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793207206_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33/.tez/application_1452793207206_0001/localmode-log-dir/dag_1452793207206_0001_1.dot
2016-01-14 17:40:07,357 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33/.tez/application_1452793207206_0001/localmode-log-dir/dag_1452793207206_0001_1-tez-dag.pb.txt
2016-01-14 17:40:07,369 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793207206_0001_1
2016-01-14 17:40:07,369 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:07,370 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33] test/(1/1) ...tform/testReplace/replace
2016-01-14 17:40:07 Running Dag: dag_1452793207206_0001_1
2016-01-14 17:40:07,381 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793207206_0001_1, submitTime=1452793207355
2016-01-14 17:40:07,381 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:07,381 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DF543B65389B44BCA5298969896EED3C
2016-01-14 17:40:07,382 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:07,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793207206_0001_1, initTime=1452793207381
2016-01-14 17:40:07,383 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:07,385 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793207206_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:07,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:DAG_STARTED]: dagID=dag_1452793207206_0001_1, startTime=1452793207385
2016-01-14 17:40:07,386 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:07,386 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793207206_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:07,387 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DF543B65389B44BCA5298969896EED3C : {17F6F863CE084AAFB52C7BFA6E84499E={InputName=17F6F863CE084AAFB52C7BFA6E84499E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:07,387 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 17F6F863CE084AAFB52C7BFA6E84499E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:07,387 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1452793207206_0001, dagName=[C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33] test/(1/1) ...tform/testReplace/replace
2016-01-14 17:40:07,386 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:07,387 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,388 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,388 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,388 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]. Starting root input initializers: 1
2016-01-14 17:40:07,389 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:07,390 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 17F6F863CE084AAFB52C7BFA6E84499E on vertex vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,392 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:07,392 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 17F6F863CE084AAFB52C7BFA6E84499E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:07,393 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:07,394 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:07,396 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 17:40:07,396 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 17F6F863CE084AAFB52C7BFA6E84499E on vertex vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,397 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:07,397 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {17F6F863CE084AAFB52C7BFA6E84499E=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:07,397 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C] parallelism set to 1
2016-01-14 17:40:07,397 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:07,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793207206_0001_1_00, eventCount=1
2016-01-14 17:40:07,398 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,398 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=99C3B2634C8E4188B9D04FAA8A0C0F8B, vertexId=vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:07,398 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=99C3B2634C8E4188B9D04FAA8A0C0F8B, vertexId=vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,400 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=99C3B2634C8E4188B9D04FAA8A0C0F8B, vertexName=DF543B65389B44BCA5298969896EED3C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:07,400 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=99C3B2634C8E4188B9D04FAA8A0C0F8B, vertexId=vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,401 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:07,402 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DF543B65389B44BCA5298969896EED3C, vertexId=vertex_1452793207206_0001_1_00, initRequestedTime=1452793207387, initedTime=1452793207401, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:07,402 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C] with distanceFromRoot: 0
2016-01-14 17:40:07,402 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:07,403 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DF543B65389B44BCA5298969896EED3C
2016-01-14 17:40:07,403 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,403 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:07,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793207206_0001_1_00, startRequestedTime=1452793207403, startedTime=1452793207403
2016-01-14 17:40:07,404 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:07,405 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:07,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:TASK_STARTED]: vertexName=DF543B65389B44BCA5298969896EED3C, taskId=task_1452793207206_0001_1_00_000000, scheduledTime=1452793207404, launchTime=1452793207404
2016-01-14 17:40:07,405 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793207206_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:07,405 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,406 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33] test/(1/1) ...tform/testReplace/replace, VertexName: DF543B65389B44BCA5298969896EED3C, VertexParallelism: 1, TaskAttemptID:attempt_1452793207206_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=17F6F863CE084AAFB52C7BFA6E84499E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=99C3B2634C8E4188B9D04FAA8A0C0F8B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:07,406 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793207206_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:07,406 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:07,407 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:07,408 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:07,408 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:07,408 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793207206_0001_00_000001
2016-01-14 17:40:07,408 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207206_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:07,409 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452793207206_0001, with dag id: dag_1452793207206_0001_1
2016-01-14 17:40:07,409 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793207206_0001_00_000001
2016-01-14 17:40:07,410 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:07,411 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:07,411 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207206_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:07,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793207206_0001_00_000001, launchTime=1452793207410
2016-01-14 17:40:07,414 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793207206_0001_00_000001
2016-01-14 17:40:07,415 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793207206_0001_1_00_000000_0] to container: [container_1452793207206_0001_00_000001]
2016-01-14 17:40:07,415 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207206_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:07,416 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793207206_0001_1_00_000000_0] started. Is using containerId: [container_1452793207206_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:07,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DF543B65389B44BCA5298969896EED3C, taskAttemptId=attempt_1452793207206_0001_1_00_000000_0, startTime=1452793207415, containerId=container_1452793207206_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793207206_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:07,416 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793207206_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:07,416 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:07,416 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793207206_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:07,416 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793207206_0001_00_000001 given task: attempt_1452793207206_0001_1_00_000000_0
2016-01-14 17:40:07,416 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793207206_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793207206_0001_1_00_000000_0
2016-01-14 17:40:07 Starting to run new task attempt: attempt_1452793207206_0001_1_00_000000_0
2016-01-14 17:40:07,417 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793207206_0001_1_00_000000_0
2016-01-14 17:40:07,417 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:07,417 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:07,417 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:07,417 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:07,417 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:07,417 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:07,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33] test/(1/1) ...tform/testReplace/replace, VertexName: DF543B65389B44BCA5298969896EED3C, VertexParallelism: 1, TaskAttemptID:attempt_1452793207206_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=17F6F863CE084AAFB52C7BFA6E84499E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=99C3B2634C8E4188B9D04FAA8A0C0F8B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:07,418 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:07,418 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793207206_0001_1_00_000000_0
2016-01-14 17:40:07,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:07,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:07,420 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793207206_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:07,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:07,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=17F6F863CE084AAFB52C7BFA6E84499E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:07,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=99C3B2634C8E4188B9D04FAA8A0C0F8B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:07,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:07,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:07,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 17F6F863CE084AAFB52C7BFA6E84499E
2016-01-14 17:40:07,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 99C3B2634C8E4188B9D04FAA8A0C0F8B
2016-01-14 17:40:07,422 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:07,423 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 99C3B2634C8E4188B9D04FAA8A0C0F8B
2016-01-14 17:40:07,422 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 17F6F863CE084AAFB52C7BFA6E84499E
2016-01-14 17:40:07,423 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:07,428 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:07,429 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 17F6F863CE084AAFB52C7BFA6E84499E
2016-01-14 17:40:07,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 17F6F863CE084AAFB52C7BFA6E84499E
2016-01-14 17:40:07,431 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DF543B65389B44BCA5298969896EED3C, ordinal: 0
2016-01-14 17:40:07,431 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF543B65389B44BCA5298969896EED3C, mem on start (mb), free: 426, total: 543, max: 672
2016-01-14 17:40:07,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:07,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:07,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:07,435 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:07,435 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 99C3B2634C8E4188B9D04FAA8A0C0F8B
2016-01-14 17:40:07,435 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:07,435 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:07,435 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:07,435 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:07,435 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:07,436 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:07,436 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:07,436 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 17:40:07,436 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 99C3B2634C8E4188B9D04FAA8A0C0F8B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:07,436 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 17F6F863CE084AAFB52C7BFA6E84499E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:07,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:07,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 17F6F863CE084AAFB52C7BFA6E84499E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:07,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:07,436 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:07,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 17F6F863CE084AAFB52C7BFA6E84499E
2016-01-14 17:40:07,437 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:07,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 17F6F863CE084AAFB52C7BFA6E84499E
2016-01-14 17:40:07,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:07,437 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793207206_0001_1_00_000000_0
2016-01-14 17:40:07,438 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:07,439 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5f0e38b2
2016-01-14 17:40:07,440 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5f0e38b2
2016-01-14 17:40:07,440 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:07,440 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:07,440 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:07,441 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 17F6F863CE084AAFB52C7BFA6E84499E
2016-01-14 17:40:07,441 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"], id: 99C3B2634C8E4188B9D04FAA8A0C0F8B
2016-01-14 17:40:07,441 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testReplace/replace"]
2016-01-14 17:40:07,442 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:07,442 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:07,442 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DF543B65389B44BCA5298969896EED3C, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:07,445 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:07,445 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:07,445 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793207206_0001_1_00_000000_0
2016-01-14 17:40:07,445 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793207206_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 17:40:07,446 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793207206_0001_1_00_000000_0
2016-01-14 17:40:07,446 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DF543B65389B44BCA5298969896EED3C, mem on close (mb), free: 424, total: 543, max: 672
2016-01-14 17:40:07 Completed running task attempt: attempt_1452793207206_0001_1_00_000000_0
2016-01-14 17:40:07,447 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793207206_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:07,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 21 [[File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10][cascading.flow.SliceCounters Process_Begin_Time=1452793207443, Process_Duration=3, Process_End_Time=1452793207446, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=10]]
2016-01-14 17:40:07,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DF543B65389B44BCA5298969896EED3C, taskAttemptId=attempt_1452793207206_0001_1_00_000000_0, startTime=1452793207415, finishTime=1452793207448, timeTaken=33, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793207443, Process_Duration=3, Process_End_Time=1452793207446, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 17:40:07,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:07,449 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793207206_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:07,449 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:07,449 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793207206_0001_1_00_000000_0
2016-01-14 17:40:07,449 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:07,450 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:07,450 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793207206_0001_00_000001
2016-01-14 17:40:07,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:07,450 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:TASK_FINISHED]: vertexName=DF543B65389B44BCA5298969896EED3C, taskId=task_1452793207206_0001_1_00_000000, startTime=1452793207415, finishTime=1452793207449, timeTaken=34, status=SUCCEEDED, successfulAttemptID=attempt_1452793207206_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793207443, Process_Duration=3, Process_End_Time=1452793207446, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 17:40:07,451 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793207206_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:07,451 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C] : 1
2016-01-14 17:40:07,451 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,452 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:07,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:VERTEX_FINISHED]: vertexName=DF543B65389B44BCA5298969896EED3C, vertexId=vertex_1452793207206_0001_1_00, initRequestedTime=1452793207387, initedTime=1452793207401, startRequestedTime=1452793207403, startedTime=1452793207403, finishTime=1452793207451, timeTaken=48, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793207443, Process_Duration=3, Process_End_Time=1452793207446, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10, vertexStats=firstTaskStartTime=1452793207415, firstTasksToStart=[ task_1452793207206_0001_1_00_000000 ], lastTaskFinishTime=1452793207448, lastTasksToFinish=[ task_1452793207206_0001_1_00_000000 ], minTaskDuration=33, maxTaskDuration=33, avgTaskDuration=33.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793207206_0001_1_00_000000 ], longestDurationTasks=[ task_1452793207206_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:07,452 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:07,452 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207206_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:07,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793207206_0001_00_000001, stoppedTime=1452793207453, exitStatus=0
2016-01-14 17:40:07,453 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207206_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:07,453 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 17:40:07,453 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793207206_0001_1
2016-01-14 17:40:07,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793207206_0001_1
2016-01-14 17:40:07,453 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 99C3B2634C8E4188B9D04FAA8A0C0F8B for vertex: vertex_1452793207206_0001_1_00 [DF543B65389B44BCA5298969896EED3C]
2016-01-14 17:40:07,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:07,461 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793207206_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:07,461 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793207206_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:07,461 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793207206_0001_00_000001, Exiting
2016-01-14 17:40:07,461 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793207206_0001_00_000001
2016-01-14 17:40:07,461 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793207206_0001_00_000001
2016-01-14 17:40:07,462 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793207206_0001_1, queueSize=0
2016-01-14 17:40:07,462 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:07,463 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793207206_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:07,463 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207206_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793207206_0001_1, startTime=1452793207385, finishTime=1452793207460, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=192, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569901056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, cascading.flow.SliceCounters, Process_Begin_Time=1452793207443, Process_Duration=3, Process_End_Time=1452793207446, Read_Duration=1, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=10
2016-01-14 17:40:07,463 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793207206_0001_00_000001 completed successfully
2016-01-14 17:40:07,463 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793207206_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:07,464 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793207206_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:07,464 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793207206_0001_00_000001
2016-01-14 17:40:07 Completed Dag: dag_1452793207206_0001_1
2016-01-14 17:40:07,464 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793207206_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:07,464 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793207206_0001_1_post
2016-01-14 17:40:07,465 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:07,465 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:07,466 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207206_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:07,471 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/testReplace/replace, applicationId=application_1452793207206_0001
2016-01-14 17:40:07,473 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:07,473 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:07,474 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:07,474 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:07,474 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:07,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:07,474 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:07,474 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:07,477 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:07,477 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:07,478 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:07,478 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:07,478 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:07,483 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C74E6B65CD1442AFB0D9611176665623/A163B6E9F3D14B1895B86B54CEFB1D33/.tez/application_1452793207206_0001
2016-01-14 17:40:07,484 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.305
2016-01-14 17:40:07,524 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testReplace/replace
2016-01-14 17:40:07,549 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:07,550 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:07,594 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 17:40:07,603 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 17:40:07,604 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:07,604 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:07,604 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:07,621 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 17:40:07,627 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2
2016-01-14 17:40:07,678 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 17:40:07,678 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:07,679 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-01-14 17:40:07,679 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 17:40:07,679 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 17:40:07,679 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 17:40:07,681 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...latform/testCount2/count2
2016-01-14 17:40:07,681 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:07,695 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 17:40:07,700 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:07,700 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:07,700 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:07,712 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0/.tez/application_1452793207700_0001 doesn't exist and is created
2016-01-14 17:40:07,729 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:07,729 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:07,729 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0/.tez/application_1452793207700_0001
2016-01-14 17:40:07,732 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793207700_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:07,750 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:07,751 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:07,753 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:07,753 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:07,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0/.tez/application_1452793207700_0001/localmode-log-dir
2016-01-14 17:40:07,754 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0/.tez/application_1452793207700_0001/localmode-log-dir/history.txt.appattempt_1452793207700_0001_000000, maxErrors=10
2016-01-14 17:40:07,754 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:07,754 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793207700_0001_000000, appSubmitTime=1452793207731, launchTime=1452793207731
2016-01-14 17:40:07,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:07,763 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:07,765 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:07,765 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:07,770 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:33135
2016-01-14 17:40:07,772 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:07,774 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793207700_0001_000000, startTime=1452793207774
2016-01-14 17:40:07,774 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:07,774 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:07,829 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:07,829 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:07,829 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1452793207700_0001, dagName=[7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0] count/(1/1) ...latform/testCount2/count2
2016-01-14 17:40:07,848 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0] count/(1/1) ...latform/testCount2/count2
2016-01-14 17:40:07,850 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793207700_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0/.tez/application_1452793207700_0001/localmode-log-dir/dag_1452793207700_0001_1.dot
2016-01-14 17:40:07,850 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0/.tez/application_1452793207700_0001/localmode-log-dir/dag_1452793207700_0001_1-tez-dag.pb.txt
2016-01-14 17:40:07,858 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793207700_0001_1
2016-01-14 17:40:07,858 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:07,859 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0] count/(1/1) ...latform/testCount2/count2
2016-01-14 17:40:07 Running Dag: dag_1452793207700_0001_1
2016-01-14 17:40:07,868 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793207700_0001_1, submitTime=1452793207848
2016-01-14 17:40:07,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:07,869 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D79FBB3FFD28444DB973C5267C755DC8
2016-01-14 17:40:07,869 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:07,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793207700_0001_1, initTime=1452793207868
2016-01-14 17:40:07,870 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:07,870 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793207700_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:07,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:DAG_STARTED]: dagID=dag_1452793207700_0001_1, startTime=1452793207870
2016-01-14 17:40:07,871 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:07,871 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793207700_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:07,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:07,871 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BDC79947B1034EC7856C7E697C356744 : {0809688DC4B240D685996C3D07A2A5D3={InputName=0809688DC4B240D685996C3D07A2A5D3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:07,871 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0809688DC4B240D685996C3D07A2A5D3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:07,871 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744]
2016-01-14 17:40:07,872 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744]
2016-01-14 17:40:07,872 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744]
2016-01-14 17:40:07,872 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1452793207700_0001, dagName=[7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0] count/(1/1) ...latform/testCount2/count2
2016-01-14 17:40:07,872 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744]. Starting root input initializers: 1
2016-01-14 17:40:07,873 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:07,873 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0809688DC4B240D685996C3D07A2A5D3 on vertex vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744]
2016-01-14 17:40:07,873 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8]
2016-01-14 17:40:07,875 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:07,875 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0809688DC4B240D685996C3D07A2A5D3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:07,876 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:07,876 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:07,877 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 17:40:07,877 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0809688DC4B240D685996C3D07A2A5D3 on vertex vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744]
2016-01-14 17:40:07,878 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:40:07,878 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8]
2016-01-14 17:40:07,878 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8]
2016-01-14 17:40:07,879 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8]
2016-01-14 17:40:07,879 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=4FA244680CC14A71A0E823564E4C068C, vertexId=vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:07,879 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=4FA244680CC14A71A0E823564E4C068C, vertexId=vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8]
2016-01-14 17:40:07,881 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=4FA244680CC14A71A0E823564E4C068C, vertexName=D79FBB3FFD28444DB973C5267C755DC8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:07,881 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=4FA244680CC14A71A0E823564E4C068C, vertexId=vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8]
2016-01-14 17:40:07,882 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:07,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D79FBB3FFD28444DB973C5267C755DC8, vertexId=vertex_1452793207700_0001_1_01, initRequestedTime=1452793207873, initedTime=1452793207882, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:40:07,883 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:40:07,883 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:07,883 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0809688DC4B240D685996C3D07A2A5D3=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:07,883 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744] parallelism set to 1
2016-01-14 17:40:07,883 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:07,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793207700_0001_1_00, eventCount=1
2016-01-14 17:40:07,885 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:07,885 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BDC79947B1034EC7856C7E697C356744, vertexId=vertex_1452793207700_0001_1_00, initRequestedTime=1452793207871, initedTime=1452793207885, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:07,886 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744] with distanceFromRoot: 0
2016-01-14 17:40:07,886 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:07,886 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BDC79947B1034EC7856C7E697C356744
2016-01-14 17:40:07,886 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744]
2016-01-14 17:40:07,887 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:07,887 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793207700_0001_1_00, startRequestedTime=1452793207886, startedTime=1452793207886
2016-01-14 17:40:07,887 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:07,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:07,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:TASK_STARTED]: vertexName=BDC79947B1034EC7856C7E697C356744, taskId=task_1452793207700_0001_1_00_000000, scheduledTime=1452793207888, launchTime=1452793207888
2016-01-14 17:40:07,888 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793207700_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:07,888 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793207700_0001_1_00 for vertex: vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8] numStartedSources: 1 numSources: 1
2016-01-14 17:40:07,889 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8] with distanceFromRoot: 1
2016-01-14 17:40:07,889 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744]
2016-01-14 17:40:07,889 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BDC79947B1034EC7856C7E697C356744 in vertex: D79FBB3FFD28444DB973C5267C755DC8
2016-01-14 17:40:07,889 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D79FBB3FFD28444DB973C5267C755DC8 with 1 source tasks and 1 pending tasks
2016-01-14 17:40:07,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:07,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793207700_0001_1_01, startRequestedTime=1452793207889, startedTime=1452793207889
2016-01-14 17:40:07,890 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:07,891 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0] count/(1/1) ...latform/testCount2/count2, VertexName: BDC79947B1034EC7856C7E697C356744, VertexParallelism: 1, TaskAttemptID:attempt_1452793207700_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0809688DC4B240D685996C3D07A2A5D3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D79FBB3FFD28444DB973C5267C755DC8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:07,891 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793207700_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:07,891 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:07,892 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:07,892 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:07,892 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:07,893 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793207700_0001_00_000001
2016-01-14 17:40:07,893 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:07,895 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793207700_0001_00_000001
2016-01-14 17:40:07,895 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:07,896 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:07,896 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:07,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793207700_0001_00_000001, launchTime=1452793207896
2016-01-14 17:40:07,897 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793207700_0001_00_000001
2016-01-14 17:40:07,898 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793207700_0001_1_00_000000_0] to container: [container_1452793207700_0001_00_000001]
2016-01-14 17:40:07,898 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:07,898 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793207700_0001_00_000001 given task: attempt_1452793207700_0001_1_00_000000_0
2016-01-14 17:40:07,898 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793207700_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793207700_0001_1_00_000000_0
2016-01-14 17:40:07,899 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452793207700_0001, with dag id: dag_1452793207700_0001_1
2016-01-14 17:40:07,900 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793207700_0001_1_00_000000_0] started. Is using containerId: [container_1452793207700_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:07 Starting to run new task attempt: attempt_1452793207700_0001_1_00_000000_0
2016-01-14 17:40:07,900 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793207700_0001_1_00_000000_0
2016-01-14 17:40:07,900 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:07,901 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:07,901 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:07,901 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:07,901 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:07,901 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:07,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0] count/(1/1) ...latform/testCount2/count2, VertexName: BDC79947B1034EC7856C7E697C356744, VertexParallelism: 1, TaskAttemptID:attempt_1452793207700_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0809688DC4B240D685996C3D07A2A5D3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D79FBB3FFD28444DB973C5267C755DC8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:07,902 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:07,902 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:07,902 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BDC79947B1034EC7856C7E697C356744, taskAttemptId=attempt_1452793207700_0001_1_00_000000_0, startTime=1452793207898, containerId=container_1452793207700_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793207700_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:07,904 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793207700_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:07,904 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793207700_0001_1_00_000000_0
2016-01-14 17:40:07,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:07,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:07,904 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793207700_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:07,906 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793207700_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:07,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:07,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0809688DC4B240D685996C3D07A2A5D3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:07,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D79FBB3FFD28444DB973C5267C755DC8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:07,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:07,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D79FBB3FFD28444DB973C5267C755DC8
2016-01-14 17:40:07,911 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:07,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:07,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0809688DC4B240D685996C3D07A2A5D3
2016-01-14 17:40:07,912 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0809688DC4B240D685996C3D07A2A5D3
2016-01-14 17:40:07,914 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:07,914 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D79FBB3FFD28444DB973C5267C755DC8
2016-01-14 17:40:07,915 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:07,917 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0809688DC4B240D685996C3D07A2A5D3
2016-01-14 17:40:07,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0809688DC4B240D685996C3D07A2A5D3
2016-01-14 17:40:07,915 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D79FBB3FFD28444DB973C5267C755DC8
2016-01-14 17:40:07,919 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BDC79947B1034EC7856C7E697C356744, ordinal: 0
2016-01-14 17:40:07,920 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDC79947B1034EC7856C7E697C356744, mem on start (mb), free: 328, total: 543, max: 672
2016-01-14 17:40:07,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:07,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:07,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:07,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:07,920 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:07,920 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:07,920 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:07,921 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:07,921 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:07,921 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:07,921 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0809688DC4B240D685996C3D07A2A5D3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:07,921 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D79FBB3FFD28444DB973C5267C755DC8, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:07,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:07,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0809688DC4B240D685996C3D07A2A5D3 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:07,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:07,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:07,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0809688DC4B240D685996C3D07A2A5D3
2016-01-14 17:40:07,923 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:07,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0809688DC4B240D685996C3D07A2A5D3
2016-01-14 17:40:07,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:07,926 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793207700_0001_1_00_000000_0
2016-01-14 17:40:07,926 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:07,927 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5adbc61e
2016-01-14 17:40:07,929 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5adbc61e
2016-01-14 17:40:07,929 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:07,930 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:07,933 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:07,934 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 0809688DC4B240D685996C3D07A2A5D3
2016-01-14 17:40:07,934 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 3C82EEED9515470197E7D53A9D1C3180
2016-01-14 17:40:07,934 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 3C82EEED9515470197E7D53A9D1C3180
2016-01-14 17:40:07,935 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@f2151d9; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@54848551; comparator=cascading.tuple.hadoop.util.TupleComparator@484cb430; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:07,935 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:07,935 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:07,936 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:07,961 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:07,961 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:07,961 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:07,961 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:07,962 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:07,963 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:07,963 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:07,964 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BDC79947B1034EC7856C7E697C356744, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:07,968 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793207700_0001_1_00_000000_0
2016-01-14 17:40:07,968 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDC79947B1034EC7856C7E697C356744, mem on close (mb), free: 409, total: 543, max: 672
2016-01-14 17:40:07,968 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:07,968 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:07,969 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 17:40:07,969 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:40:07,972 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:07 Completed running task attempt: attempt_1452793207700_0001_1_00_000000_0
2016-01-14 17:40:07,976 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793207700_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:07,976 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793207700_0001_1_00, eventCount=1
2016-01-14 17:40:07,977 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:07,977 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BDC79947B1034EC7856C7E697C356744, taskAttemptId=attempt_1452793207700_0001_1_00_000000_0, startTime=1452793207898, finishTime=1452793207976, timeTaken=78, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793207964, Process_Duration=4, Process_End_Time=1452793207968, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:40:07,977 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793207700_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:07,977 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793207700_0001_1_00_000000_0
2016-01-14 17:40:07,977 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:07,979 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:07,979 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:TASK_FINISHED]: vertexName=BDC79947B1034EC7856C7E697C356744, taskId=task_1452793207700_0001_1_00_000000, startTime=1452793207898, finishTime=1452793207977, timeTaken=79, status=SUCCEEDED, successfulAttemptID=attempt_1452793207700_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793207964, Process_Duration=4, Process_End_Time=1452793207968, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:40:07,979 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793207700_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:07,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793207964, Process_Duration=4, Process_End_Time=1452793207968, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:40:07,980 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:07,980 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:07,980 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793207700_0001_00_000001
2016-01-14 17:40:07,980 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744] : 1
2016-01-14 17:40:07,980 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744]
2016-01-14 17:40:07,981 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:07,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:VERTEX_FINISHED]: vertexName=BDC79947B1034EC7856C7E697C356744, vertexId=vertex_1452793207700_0001_1_00, initRequestedTime=1452793207871, initedTime=1452793207885, startRequestedTime=1452793207886, startedTime=1452793207886, finishTime=1452793207980, timeTaken=94, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=1015, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793207964, Process_Duration=4, Process_End_Time=1452793207968, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452793207898, firstTasksToStart=[ task_1452793207700_0001_1_00_000000 ], lastTaskFinishTime=1452793207976, lastTasksToFinish=[ task_1452793207700_0001_1_00_000000 ], minTaskDuration=78, maxTaskDuration=78, avgTaskDuration=78.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793207700_0001_1_00_000000 ], longestDurationTasks=[ task_1452793207700_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:07,982 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:07,982 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:07,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793207700_0001_00_000001, stoppedTime=1452793207982, exitStatus=0
2016-01-14 17:40:07,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:07,982 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:07,983 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8] attempt: attempt_1452793207700_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:07,983 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D79FBB3FFD28444DB973C5267C755DC8
2016-01-14 17:40:07,983 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8]
2016-01-14 17:40:07,983 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:07,984 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793207700_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:40:07,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:TASK_STARTED]: vertexName=D79FBB3FFD28444DB973C5267C755DC8, taskId=task_1452793207700_0001_1_01_000000, scheduledTime=1452793207984, launchTime=1452793207984
2016-01-14 17:40:07,985 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793207700_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:07,985 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:07,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:07,986 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8]
2016-01-14 17:40:07,986 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0] count/(1/1) ...latform/testCount2/count2, VertexName: D79FBB3FFD28444DB973C5267C755DC8, VertexParallelism: 1, TaskAttemptID:attempt_1452793207700_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BDC79947B1034EC7856C7E697C356744, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4FA244680CC14A71A0E823564E4C068C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:07,987 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793207700_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:07,987 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:07,987 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793207700_0001_00_000002
2016-01-14 17:40:07,988 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:07,990 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793207700_0001_00_000002
2016-01-14 17:40:07,990 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:07,990 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793207700_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:07,991 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793207700_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:07,991 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793207700_0001_00_000001, Exiting
2016-01-14 17:40:07,991 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793207700_0001_00_000001
2016-01-14 17:40:07,991 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793207700_0001_00_000001
2016-01-14 17:40:07,993 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:07,993 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:07,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793207700_0001_00_000002, launchTime=1452793207992
2016-01-14 17:40:07,994 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793207700_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:07,994 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793207700_0001_00_000001 completed successfully
2016-01-14 17:40:07,994 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793207700_0001_00_000002
2016-01-14 17:40:07,994 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:07,995 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793207700_0001_1_01_000000_0] to container: [container_1452793207700_0001_00_000002]
2016-01-14 17:40:07,995 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:07,995 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793207700_0001_00_000002 given task: attempt_1452793207700_0001_1_01_000000_0
2016-01-14 17:40:07,995 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793207700_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793207700_0001_1_01_000000_0
2016-01-14 17:40:07,996 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793207700_0001_1_01_000000_0] started. Is using containerId: [container_1452793207700_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:40:07,996 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:07 Starting to run new task attempt: attempt_1452793207700_0001_1_01_000000_0
2016-01-14 17:40:07,996 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793207700_0001_1_01_000000_0
2016-01-14 17:40:07,997 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:07,997 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:07,997 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:07,997 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:07,998 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:07,998 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:07,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D79FBB3FFD28444DB973C5267C755DC8, taskAttemptId=attempt_1452793207700_0001_1_01_000000_0, startTime=1452793207995, containerId=container_1452793207700_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793207700_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:40:07,998 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793207700_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:07,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0] count/(1/1) ...latform/testCount2/count2, VertexName: D79FBB3FFD28444DB973C5267C755DC8, VertexParallelism: 1, TaskAttemptID:attempt_1452793207700_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BDC79947B1034EC7856C7E697C356744, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4FA244680CC14A71A0E823564E4C068C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:07,999 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793207700_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:07,999 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:08,000 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793207700_0001_1_01_000000_0
2016-01-14 17:40:08,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:08,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:08,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:08,002 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793207700_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:40:08,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=4FA244680CC14A71A0E823564E4C068C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:08,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:08,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 4FA244680CC14A71A0E823564E4C068C
2016-01-14 17:40:08,004 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:08,004 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 4FA244680CC14A71A0E823564E4C068C
2016-01-14 17:40:08,006 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:08,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BDC79947B1034EC7856C7E697C356744, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:08,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:08,008 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BDC79947B1034EC7856C7E697C356744
2016-01-14 17:40:08,010 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:08,010 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:08,010 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BDC79947B1034EC7856C7E697C356744
2016-01-14 17:40:08,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BDC79947B1034EC7856C7E697C356744
2016-01-14 17:40:08,013 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:08,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 4FA244680CC14A71A0E823564E4C068C
2016-01-14 17:40:08,013 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D79FBB3FFD28444DB973C5267C755DC8, ordinal: 1
2016-01-14 17:40:08,014 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D79FBB3FFD28444DB973C5267C755DC8, mem on start (mb), free: 403, total: 543, max: 672
2016-01-14 17:40:08,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:08,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:08,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:08,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:08,014 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:08,014 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:08,015 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:08,015 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:08,015 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:08,015 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:40:08,016 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 4FA244680CC14A71A0E823564E4C068C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:08,016 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BDC79947B1034EC7856C7E697C356744, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:40:08,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:08,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BDC79947B1034EC7856C7E697C356744 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:08,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:08,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:08,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BDC79947B1034EC7856C7E697C356744
2016-01-14 17:40:08,018 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:08,018 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:08,019 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BDC79947B1034EC7856C7E697C356744 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:08,019 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:40:08,019 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:08,019 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:08,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BDC79947B1034EC7856C7E697C356744
2016-01-14 17:40:08,021 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:08,021 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793207700_0001_1_01_000000_0
2016-01-14 17:40:08,025 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 17:40:08,025 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793207700_0001_1_00_000000_0_10068, runDuration: 62528, ]
2016-01-14 17:40:08,025 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 3C82EEED9515470197E7D53A9D1C3180
2016-01-14 17:40:08,026 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"], id: 4FA244680CC14A71A0E823564E4C068C
2016-01-14 17:40:08,026 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount2/count2"]
2016-01-14 17:40:08,026 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 3C82EEED9515470197E7D53A9D1C3180, for 1 inputs
2016-01-14 17:40:08,027 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793207700_0001_1_00_000000_0_10068] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452793208027, TimeTaken=1, Rate=0.44 MB/s
2016-01-14 17:40:08,027 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BDC79947B1034EC7856C7E697C356744
2016-01-14 17:40:08,027 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:08,027 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BDC79947B1034EC7856C7E697C356744] #11 in 1ms
2016-01-14 17:40:08,027 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BDC79947B1034EC7856C7E697C356744] #11, status:false, isInterrupted:false
2016-01-14 17:40:08,028 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:08,028 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 17:40:08,029 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:08,029 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:08,029 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 17:40:08,029 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BDC79947B1034EC7856C7E697C356744
2016-01-14 17:40:08,029 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:08,030 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D79FBB3FFD28444DB973C5267C755DC8, all 1 inputs ready in: 00:00:00.003
2016-01-14 17:40:08,030 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@17bd11ee; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7de42c6e
2016-01-14 17:40:08,033 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:08,033 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:08,034 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793207700_0001_1_01_000000_0
2016-01-14 17:40:08,034 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793207700_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:40:08,035 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793207700_0001_1_01_000000_0
2016-01-14 17:40:08,035 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D79FBB3FFD28444DB973C5267C755DC8, mem on close (mb), free: 400, total: 543, max: 672
2016-01-14 17:40:08,035 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BDC79947B1034EC7856C7E697C356744
2016-01-14 17:40:08,036 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:08,036 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:08,036 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:08,037 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:08,037 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:08 Completed running task attempt: attempt_1452793207700_0001_1_01_000000_0
2016-01-14 17:40:08,038 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793207700_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:08,039 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793208030, Process_Duration=5, Process_End_Time=1452793208035, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 17:40:08,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D79FBB3FFD28444DB973C5267C755DC8, taskAttemptId=attempt_1452793207700_0001_1_01_000000_0, startTime=1452793207995, finishTime=1452793208039, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793208030, Process_Duration=5, Process_End_Time=1452793208035, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 17:40:08,039 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793207700_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:08,040 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793207700_0001_1_01_000000_0
2016-01-14 17:40:08,040 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:08,040 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:08,040 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:TASK_FINISHED]: vertexName=D79FBB3FFD28444DB973C5267C755DC8, taskId=task_1452793207700_0001_1_01_000000, startTime=1452793207995, finishTime=1452793208040, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452793207700_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793208030, Process_Duration=5, Process_End_Time=1452793208035, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 17:40:08,041 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:08,041 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793207700_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:08,041 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8] : 1
2016-01-14 17:40:08,041 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8]
2016-01-14 17:40:08,042 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:08,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:VERTEX_FINISHED]: vertexName=D79FBB3FFD28444DB973C5267C755DC8, vertexId=vertex_1452793207700_0001_1_01, initRequestedTime=1452793207873, initedTime=1452793207882, startRequestedTime=1452793207889, startedTime=1452793207889, finishTime=1452793208041, timeTaken=152, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=569376768, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793208030, Process_Duration=5, Process_End_Time=1452793208035, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452793207995, firstTasksToStart=[ task_1452793207700_0001_1_01_000000 ], lastTaskFinishTime=1452793208039, lastTasksToFinish=[ task_1452793207700_0001_1_01_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793207700_0001_1_01_000000 ], longestDurationTasks=[ task_1452793207700_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:08,042 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:08,043 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:08,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793207700_0001_00_000002, stoppedTime=1452793208043, exitStatus=0
2016-01-14 17:40:08,043 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:08,043 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:08,043 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793207700_0001_1
2016-01-14 17:40:08,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793207700_0001_1
2016-01-14 17:40:08,043 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 4FA244680CC14A71A0E823564E4C068C for vertex: vertex_1452793207700_0001_1_01 [D79FBB3FFD28444DB973C5267C755DC8]
2016-01-14 17:40:08,043 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:08,044 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:08,044 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:08,045 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793207700_0001_00_000002
2016-01-14 17:40:08,045 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793207700_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:40:08,045 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793207700_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:08,046 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793207700_0001_00_000002, Exiting
2016-01-14 17:40:08,046 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793207700_0001_00_000002
2016-01-14 17:40:08,046 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793207700_0001_00_000002
2016-01-14 17:40:08,046 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793207700_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:08,046 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793207700_0001_00_000002 completed successfully
2016-01-14 17:40:08,050 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793207700_0001_1_00 [BDC79947B1034EC7856C7E697C356744]
2016-01-14 17:40:08,052 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793207700_0001_1, queueSize=0
2016-01-14 17:40:08,053 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:08,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793207700_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793207700_0001_1, startTime=1452793207870, finishTime=1452793208050, timeTaken=180, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=1297, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1138753536, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905586415994, Process_Duration=9, Process_End_Time=2905586416003, Read_Duration=3, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 17:40:08,053 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793207700_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:08,054 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793207700_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:08,054 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793207700_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:08,054 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793207700_0001_00_000002
2016-01-14 17:40:08 Completed Dag: dag_1452793207700_0001_1
2016-01-14 17:40:08,054 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793207700_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:08,054 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793207700_0001_1_post
2016-01-14 17:40:08,054 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:08,054 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:08,063 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount2/count2, applicationId=application_1452793207700_0001
2016-01-14 17:40:08,067 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:08,067 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:08,067 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:08,067 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:08,067 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:08,068 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:08,069 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:08,069 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:08,070 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:08,070 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:08,070 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:08,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:08,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:08,075 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7EFF589BA44E4842AE942346C4AF628B/D89C06A0DC894584A882E5F1BC8593A0/.tez/application_1452793207700_0001
2016-01-14 17:40:08,076 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.397
2016-01-14 17:40:08,159 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount2/count2
2016-01-14 17:40:08,185 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:08,186 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:08,230 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 17:40:08,238 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-01-14 17:40:08,238 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:08,239 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:08,239 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:08,256 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 17:40:08,262 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3
2016-01-14 17:40:08,313 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 17:40:08,314 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:08,314 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-01-14 17:40:08,314 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 17:40:08,314 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 17:40:08,315 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 17:40:08,315 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...latform/testCount3/count3
2016-01-14 17:40:08,315 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:08,330 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 17:40:08,335 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:08,336 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:08,336 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:08,345 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494/.tez/application_1452793208336_0001 doesn't exist and is created
2016-01-14 17:40:08,360 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:08,360 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:08,361 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494/.tez/application_1452793208336_0001
2016-01-14 17:40:08,364 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793208336_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:08,375 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:08,376 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:08,378 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:08,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:08,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494/.tez/application_1452793208336_0001/localmode-log-dir
2016-01-14 17:40:08,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494/.tez/application_1452793208336_0001/localmode-log-dir/history.txt.appattempt_1452793208336_0001_000000, maxErrors=10
2016-01-14 17:40:08,379 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:08,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793208336_0001_000000, appSubmitTime=1452793208363, launchTime=1452793208363
2016-01-14 17:40:08,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:08,386 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:08,386 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:08,389 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:08,392 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:50734
2016-01-14 17:40:08,393 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:08,397 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793208336_0001_000000, startTime=1452793208397
2016-01-14 17:40:08,397 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:08,397 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:08,460 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:08,460 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:08,460 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1452793208336_0001, dagName=[2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494] count/(1/1) ...latform/testCount3/count3
2016-01-14 17:40:08,482 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494] count/(1/1) ...latform/testCount3/count3
2016-01-14 17:40:08,483 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793208336_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494/.tez/application_1452793208336_0001/localmode-log-dir/dag_1452793208336_0001_1.dot
2016-01-14 17:40:08,484 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494/.tez/application_1452793208336_0001/localmode-log-dir/dag_1452793208336_0001_1-tez-dag.pb.txt
2016-01-14 17:40:08,492 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793208336_0001_1
2016-01-14 17:40:08,492 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:08,492 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494] count/(1/1) ...latform/testCount3/count3
2016-01-14 17:40:08 Running Dag: dag_1452793208336_0001_1
2016-01-14 17:40:08,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793208336_0001_1, submitTime=1452793208482
2016-01-14 17:40:08,501 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:08,502 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FF9E8A6F8FDB40AEBC8EC81D5338669E
2016-01-14 17:40:08,502 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:08,503 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:08,502 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793208336_0001_1, initTime=1452793208502
2016-01-14 17:40:08,503 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793208336_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:08,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:DAG_STARTED]: dagID=dag_1452793208336_0001_1, startTime=1452793208503
2016-01-14 17:40:08,503 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:08,503 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793208336_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:08,503 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ECDAAA0E643E459C8EBF72D0ECEA462A : {FA8494D62B2E4E02BF63179E1DC40AC4={InputName=FA8494D62B2E4E02BF63179E1DC40AC4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:08,503 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: FA8494D62B2E4E02BF63179E1DC40AC4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:08,504 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A]
2016-01-14 17:40:08,504 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A]
2016-01-14 17:40:08,504 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A]
2016-01-14 17:40:08,504 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A]. Starting root input initializers: 1
2016-01-14 17:40:08,503 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:08,504 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:08,505 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E]
2016-01-14 17:40:08,505 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1452793208336_0001, dagName=[2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494] count/(1/1) ...latform/testCount3/count3
2016-01-14 17:40:08,506 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: FA8494D62B2E4E02BF63179E1DC40AC4 on vertex vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A]
2016-01-14 17:40:08,508 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:40:08,508 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E]
2016-01-14 17:40:08,509 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E]
2016-01-14 17:40:08,509 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E]
2016-01-14 17:40:08,509 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2181E041491F4613B8053CD7D894A90F, vertexId=vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:08,509 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2181E041491F4613B8053CD7D894A90F, vertexId=vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E]
2016-01-14 17:40:08,509 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:08,510 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input FA8494D62B2E4E02BF63179E1DC40AC4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:08,511 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:08,511 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:08,513 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2181E041491F4613B8053CD7D894A90F, vertexName=FF9E8A6F8FDB40AEBC8EC81D5338669E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:08,513 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2181E041491F4613B8053CD7D894A90F, vertexId=vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E]
2016-01-14 17:40:08,514 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 17:40:08,514 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: FA8494D62B2E4E02BF63179E1DC40AC4 on vertex vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A]
2016-01-14 17:40:08,515 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:08,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FF9E8A6F8FDB40AEBC8EC81D5338669E, vertexId=vertex_1452793208336_0001_1_01, initRequestedTime=1452793208505, initedTime=1452793208514, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:40:08,515 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:40:08,515 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:08,515 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {FA8494D62B2E4E02BF63179E1DC40AC4=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:08,516 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A] parallelism set to 1
2016-01-14 17:40:08,516 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:08,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793208336_0001_1_00, eventCount=1
2016-01-14 17:40:08,517 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:08,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ECDAAA0E643E459C8EBF72D0ECEA462A, vertexId=vertex_1452793208336_0001_1_00, initRequestedTime=1452793208503, initedTime=1452793208517, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:08,518 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A] with distanceFromRoot: 0
2016-01-14 17:40:08,518 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:08,518 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ECDAAA0E643E459C8EBF72D0ECEA462A
2016-01-14 17:40:08,519 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A]
2016-01-14 17:40:08,519 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:08,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793208336_0001_1_00, startRequestedTime=1452793208518, startedTime=1452793208518
2016-01-14 17:40:08,520 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:08,520 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:08,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:TASK_STARTED]: vertexName=ECDAAA0E643E459C8EBF72D0ECEA462A, taskId=task_1452793208336_0001_1_00_000000, scheduledTime=1452793208520, launchTime=1452793208520
2016-01-14 17:40:08,521 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208336_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:08,521 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793208336_0001_1_00 for vertex: vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E] numStartedSources: 1 numSources: 1
2016-01-14 17:40:08,521 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E] with distanceFromRoot: 1
2016-01-14 17:40:08,521 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A]
2016-01-14 17:40:08,522 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: ECDAAA0E643E459C8EBF72D0ECEA462A in vertex: FF9E8A6F8FDB40AEBC8EC81D5338669E
2016-01-14 17:40:08,522 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FF9E8A6F8FDB40AEBC8EC81D5338669E with 1 source tasks and 1 pending tasks
2016-01-14 17:40:08,522 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:08,524 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:40:08,524 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:40:08,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793208336_0001_1_01, startRequestedTime=1452793208522, startedTime=1452793208522
2016-01-14 17:40:08,525 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:08,525 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494] count/(1/1) ...latform/testCount3/count3, VertexName: ECDAAA0E643E459C8EBF72D0ECEA462A, VertexParallelism: 1, TaskAttemptID:attempt_1452793208336_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FA8494D62B2E4E02BF63179E1DC40AC4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FF9E8A6F8FDB40AEBC8EC81D5338669E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:08,525 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208336_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:08,526 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:08,526 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:08,528 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:08,528 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:08,528 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793208336_0001_00_000001
2016-01-14 17:40:08,528 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:08,530 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452793208336_0001, with dag id: dag_1452793208336_0001_1
2016-01-14 17:40:08,532 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793208336_0001_00_000001
2016-01-14 17:40:08,532 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:08,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:08,533 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:08,534 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793208336_0001_00_000001, launchTime=1452793208533
2016-01-14 17:40:08,538 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793208336_0001_00_000001
2016-01-14 17:40:08,538 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793208336_0001_1_00_000000_0] to container: [container_1452793208336_0001_00_000001]
2016-01-14 17:40:08,538 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:08,538 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793208336_0001_00_000001 given task: attempt_1452793208336_0001_1_00_000000_0
2016-01-14 17:40:08,539 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793208336_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793208336_0001_1_00_000000_0
2016-01-14 17:40:08,539 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793208336_0001_1_00_000000_0] started. Is using containerId: [container_1452793208336_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:08,540 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ECDAAA0E643E459C8EBF72D0ECEA462A, taskAttemptId=attempt_1452793208336_0001_1_00_000000_0, startTime=1452793208539, containerId=container_1452793208336_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793208336_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:08,540 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:08,540 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208336_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:08,540 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208336_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:08 Starting to run new task attempt: attempt_1452793208336_0001_1_00_000000_0
2016-01-14 17:40:08,541 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793208336_0001_1_00_000000_0
2016-01-14 17:40:08,541 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:08,541 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:08,542 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:08,542 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:08,542 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:08,542 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:08,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494] count/(1/1) ...latform/testCount3/count3, VertexName: ECDAAA0E643E459C8EBF72D0ECEA462A, VertexParallelism: 1, TaskAttemptID:attempt_1452793208336_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FA8494D62B2E4E02BF63179E1DC40AC4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FF9E8A6F8FDB40AEBC8EC81D5338669E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:08,543 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:08,544 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793208336_0001_1_00_000000_0
2016-01-14 17:40:08,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:08,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:08,545 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FA8494D62B2E4E02BF63179E1DC40AC4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:08,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:08,546 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793208336_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:08,547 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FA8494D62B2E4E02BF63179E1DC40AC4
2016-01-14 17:40:08,548 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: FA8494D62B2E4E02BF63179E1DC40AC4
2016-01-14 17:40:08,549 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:08,550 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:08,550 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: FA8494D62B2E4E02BF63179E1DC40AC4
2016-01-14 17:40:08,551 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FA8494D62B2E4E02BF63179E1DC40AC4
2016-01-14 17:40:08,552 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:08,555 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FF9E8A6F8FDB40AEBC8EC81D5338669E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:08,555 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:08,555 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FF9E8A6F8FDB40AEBC8EC81D5338669E
2016-01-14 17:40:08,570 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:08,570 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FF9E8A6F8FDB40AEBC8EC81D5338669E
2016-01-14 17:40:08,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FF9E8A6F8FDB40AEBC8EC81D5338669E
2016-01-14 17:40:08,573 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ECDAAA0E643E459C8EBF72D0ECEA462A, ordinal: 0
2016-01-14 17:40:08,573 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ECDAAA0E643E459C8EBF72D0ECEA462A, mem on start (mb), free: 506, total: 540, max: 672
2016-01-14 17:40:08,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:08,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:08,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:08,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:08,573 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:08,574 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:08,574 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:08,574 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:08,574 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:08,574 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:08,575 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FA8494D62B2E4E02BF63179E1DC40AC4, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:08,575 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FF9E8A6F8FDB40AEBC8EC81D5338669E, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:08,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:08,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FA8494D62B2E4E02BF63179E1DC40AC4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:08,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:08,576 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:08,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FA8494D62B2E4E02BF63179E1DC40AC4
2016-01-14 17:40:08,576 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:08,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FA8494D62B2E4E02BF63179E1DC40AC4
2016-01-14 17:40:08,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:08,577 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793208336_0001_1_00_000000_0
2016-01-14 17:40:08,577 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:08,578 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@564686ea
2016-01-14 17:40:08,579 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@564686ea
2016-01-14 17:40:08,579 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:08,579 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:08,580 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:08,581 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: FA8494D62B2E4E02BF63179E1DC40AC4
2016-01-14 17:40:08,581 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: 773457D9F310411297832158E67413B0
2016-01-14 17:40:08,581 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] 773457D9F310411297832158E67413B0
2016-01-14 17:40:08,582 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@11de5b13; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1d2d458f; comparator=cascading.tuple.hadoop.util.TupleComparator@731be40; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:08,583 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:08,583 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:08,583 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:08,596 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:08,596 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:08,597 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:08,597 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:08,597 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:08,598 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:08,598 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:08,599 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ECDAAA0E643E459C8EBF72D0ECEA462A, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:08,601 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793208336_0001_1_00_000000_0
2016-01-14 17:40:08,601 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ECDAAA0E643E459C8EBF72D0ECEA462A, mem on close (mb), free: 405, total: 540, max: 672
2016-01-14 17:40:08,602 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:08,602 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:08,602 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 17:40:08,602 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:40:08,605 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:08 Completed running task attempt: attempt_1452793208336_0001_1_00_000000_0
2016-01-14 17:40:08,607 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793208336_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:08,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793208336_0001_1_00, eventCount=1
2016-01-14 17:40:08,609 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:08,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ECDAAA0E643E459C8EBF72D0ECEA462A, taskAttemptId=attempt_1452793208336_0001_1_00_000000_0, startTime=1452793208539, finishTime=1452793208608, timeTaken=69, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=566231040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793208599, Process_Duration=2, Process_End_Time=1452793208601, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:40:08,609 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208336_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:08,609 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793208336_0001_1_00_000000_0
2016-01-14 17:40:08,609 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:08,610 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:08,610 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:TASK_FINISHED]: vertexName=ECDAAA0E643E459C8EBF72D0ECEA462A, taskId=task_1452793208336_0001_1_00_000000, startTime=1452793208539, finishTime=1452793208609, timeTaken=70, status=SUCCEEDED, successfulAttemptID=attempt_1452793208336_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=566231040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793208599, Process_Duration=2, Process_End_Time=1452793208601, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:40:08,610 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208336_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:08,610 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A] : 1
2016-01-14 17:40:08,610 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A]
2016-01-14 17:40:08,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:VERTEX_FINISHED]: vertexName=ECDAAA0E643E459C8EBF72D0ECEA462A, vertexId=vertex_1452793208336_0001_1_00, initRequestedTime=1452793208503, initedTime=1452793208517, startRequestedTime=1452793208518, startedTime=1452793208518, finishTime=1452793208611, timeTaken=93, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=566231040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793208599, Process_Duration=2, Process_End_Time=1452793208601, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452793208539, firstTasksToStart=[ task_1452793208336_0001_1_00_000000 ], lastTaskFinishTime=1452793208608, lastTasksToFinish=[ task_1452793208336_0001_1_00_000000 ], minTaskDuration=69, maxTaskDuration=69, avgTaskDuration=69.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793208336_0001_1_00_000000 ], longestDurationTasks=[ task_1452793208336_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:08,612 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:08,612 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:08,613 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:08,613 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=566231040, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793208599, Process_Duration=2, Process_End_Time=1452793208601, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:40:08,614 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:08,614 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:08,614 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793208336_0001_00_000001
2016-01-14 17:40:08,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793208336_0001_00_000001, stoppedTime=1452793208613, exitStatus=0
2016-01-14 17:40:08,614 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793208336_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:08,614 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793208336_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:08,614 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:08,615 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793208336_0001_00_000001, Exiting
2016-01-14 17:40:08,615 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793208336_0001_00_000001
2016-01-14 17:40:08,615 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793208336_0001_00_000001
2016-01-14 17:40:08,614 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:08,616 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E] attempt: attempt_1452793208336_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:08,616 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FF9E8A6F8FDB40AEBC8EC81D5338669E
2016-01-14 17:40:08,616 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E]
2016-01-14 17:40:08,617 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:08,617 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793208336_0001_00_000001
2016-01-14 17:40:08,617 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793208336_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:08,617 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:08,617 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:TASK_STARTED]: vertexName=FF9E8A6F8FDB40AEBC8EC81D5338669E, taskId=task_1452793208336_0001_1_01_000000, scheduledTime=1452793208617, launchTime=1452793208617
2016-01-14 17:40:08,618 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208336_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:08,618 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:08,618 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E]
2016-01-14 17:40:08,618 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494] count/(1/1) ...latform/testCount3/count3, VertexName: FF9E8A6F8FDB40AEBC8EC81D5338669E, VertexParallelism: 1, TaskAttemptID:attempt_1452793208336_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ECDAAA0E643E459C8EBF72D0ECEA462A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2181E041491F4613B8053CD7D894A90F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:08,618 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208336_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:08,618 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793208336_0001_00_000001 completed successfully
2016-01-14 17:40:08,619 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:08,619 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:08,619 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793208336_0001_00_000002
2016-01-14 17:40:08,619 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:08,621 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793208336_0001_00_000002
2016-01-14 17:40:08,622 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:40:08,625 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:08,626 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:08,626 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793208336_0001_00_000002, launchTime=1452793208626
2016-01-14 17:40:08,627 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793208336_0001_00_000002
2016-01-14 17:40:08,627 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793208336_0001_1_01_000000_0] to container: [container_1452793208336_0001_00_000002]
2016-01-14 17:40:08,627 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:08,627 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793208336_0001_00_000002 given task: attempt_1452793208336_0001_1_01_000000_0
2016-01-14 17:40:08,628 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793208336_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793208336_0001_1_01_000000_0
2016-01-14 17:40:08,628 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793208336_0001_1_01_000000_0] started. Is using containerId: [container_1452793208336_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:40:08,628 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FF9E8A6F8FDB40AEBC8EC81D5338669E, taskAttemptId=attempt_1452793208336_0001_1_01_000000_0, startTime=1452793208628, containerId=container_1452793208336_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793208336_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:40:08,628 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:08,628 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208336_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:08 Starting to run new task attempt: attempt_1452793208336_0001_1_01_000000_0
2016-01-14 17:40:08,629 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208336_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:08,629 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793208336_0001_1_01_000000_0
2016-01-14 17:40:08,629 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:08,629 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:08,629 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:08,630 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:08,630 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:08,630 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:08,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494] count/(1/1) ...latform/testCount3/count3, VertexName: FF9E8A6F8FDB40AEBC8EC81D5338669E, VertexParallelism: 1, TaskAttemptID:attempt_1452793208336_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=ECDAAA0E643E459C8EBF72D0ECEA462A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2181E041491F4613B8053CD7D894A90F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:08,631 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:08,631 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793208336_0001_1_01_000000_0
2016-01-14 17:40:08,631 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:08,632 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:08,632 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793208336_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:40:08,632 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=ECDAAA0E643E459C8EBF72D0ECEA462A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:08,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2181E041491F4613B8053CD7D894A90F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:08,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:08,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: ECDAAA0E643E459C8EBF72D0ECEA462A
2016-01-14 17:40:08,632 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:08,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:08,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2181E041491F4613B8053CD7D894A90F
2016-01-14 17:40:08,634 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:08,634 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2181E041491F4613B8053CD7D894A90F
2016-01-14 17:40:08,636 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:08,636 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:08,636 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: ECDAAA0E643E459C8EBF72D0ECEA462A
2016-01-14 17:40:08,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: ECDAAA0E643E459C8EBF72D0ECEA462A
2016-01-14 17:40:08,638 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:08,644 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:08,644 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2181E041491F4613B8053CD7D894A90F
2016-01-14 17:40:08,645 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FF9E8A6F8FDB40AEBC8EC81D5338669E, ordinal: 1
2016-01-14 17:40:08,645 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF9E8A6F8FDB40AEBC8EC81D5338669E, mem on start (mb), free: 398, total: 540, max: 672
2016-01-14 17:40:08,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:08,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:08,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:08,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:08,646 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:08,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:08,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:08,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:08,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:08,647 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:40:08,647 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2181E041491F4613B8053CD7D894A90F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:08,647 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, ECDAAA0E643E459C8EBF72D0ECEA462A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:40:08,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:08,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: ECDAAA0E643E459C8EBF72D0ECEA462A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:08,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: ECDAAA0E643E459C8EBF72D0ECEA462A
2016-01-14 17:40:08,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:08,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:08,648 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:08,649 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:08,649 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: ECDAAA0E643E459C8EBF72D0ECEA462A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:08,649 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:40:08,649 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:08,649 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:08,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: ECDAAA0E643E459C8EBF72D0ECEA462A
2016-01-14 17:40:08,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:08,651 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793208336_0001_1_01_000000_0
2016-01-14 17:40:08,654 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 17:40:08,654 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793208336_0001_1_00_000000_0_10074, runDuration: 50472, ]
2016-01-14 17:40:08,654 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: 773457D9F310411297832158E67413B0
2016-01-14 17:40:08,655 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"], id: 2181E041491F4613B8053CD7D894A90F
2016-01-14 17:40:08,655 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount3/count3"]
2016-01-14 17:40:08,656 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] 773457D9F310411297832158E67413B0, for 1 inputs
2016-01-14 17:40:08,656 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793208336_0001_1_00_000000_0_10074] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452793208656, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 17:40:08,657 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : ECDAAA0E643E459C8EBF72D0ECEA462A
2016-01-14 17:40:08,657 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:08,657 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [ECDAAA0E643E459C8EBF72D0ECEA462A] #12 in 2ms
2016-01-14 17:40:08,657 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [ECDAAA0E643E459C8EBF72D0ECEA462A] #12, status:false, isInterrupted:false
2016-01-14 17:40:08,658 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:08,658 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 17:40:08,658 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:08,658 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:08,659 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 17:40:08,659 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : ECDAAA0E643E459C8EBF72D0ECEA462A
2016-01-14 17:40:08,659 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:08,659 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FF9E8A6F8FDB40AEBC8EC81D5338669E, all 1 inputs ready in: 00:00:00.003
2016-01-14 17:40:08,660 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@47e061d5; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5c370bd0
2016-01-14 17:40:08,664 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:08,664 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:08,665 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793208336_0001_1_01_000000_0
2016-01-14 17:40:08,665 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793208336_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:40:08,666 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793208336_0001_1_01_000000_0
2016-01-14 17:40:08,666 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FF9E8A6F8FDB40AEBC8EC81D5338669E, mem on close (mb), free: 396, total: 540, max: 672
2016-01-14 17:40:08,666 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: ECDAAA0E643E459C8EBF72D0ECEA462A
2016-01-14 17:40:08,667 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:08,668 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:08,668 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:08,668 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:08,668 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:08 Completed running task attempt: attempt_1452793208336_0001_1_01_000000_0
2016-01-14 17:40:08,669 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793208336_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:08,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=566231040, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793208660, Process_Duration=6, Process_End_Time=1452793208666, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 17:40:08,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FF9E8A6F8FDB40AEBC8EC81D5338669E, taskAttemptId=attempt_1452793208336_0001_1_01_000000_0, startTime=1452793208628, finishTime=1452793208674, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=566231040, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793208660, Process_Duration=6, Process_End_Time=1452793208666, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 17:40:08,674 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:08,674 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:08,674 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208336_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:08,675 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:08,675 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793208336_0001_1_01_000000_0
2016-01-14 17:40:08,675 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:08,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:TASK_FINISHED]: vertexName=FF9E8A6F8FDB40AEBC8EC81D5338669E, taskId=task_1452793208336_0001_1_01_000000, startTime=1452793208628, finishTime=1452793208675, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1452793208336_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=566231040, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793208660, Process_Duration=6, Process_End_Time=1452793208666, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 17:40:08,676 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208336_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:08,676 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E] : 1
2016-01-14 17:40:08,676 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E]
2016-01-14 17:40:08,676 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:08,677 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:08,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:VERTEX_FINISHED]: vertexName=FF9E8A6F8FDB40AEBC8EC81D5338669E, vertexId=vertex_1452793208336_0001_1_01, initRequestedTime=1452793208505, initedTime=1452793208514, startRequestedTime=1452793208522, startedTime=1452793208522, finishTime=1452793208676, timeTaken=154, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=566231040, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793208660, Process_Duration=6, Process_End_Time=1452793208666, Read_Duration=1, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452793208628, firstTasksToStart=[ task_1452793208336_0001_1_01_000000 ], lastTaskFinishTime=1452793208674, lastTasksToFinish=[ task_1452793208336_0001_1_01_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793208336_0001_1_01_000000 ], longestDurationTasks=[ task_1452793208336_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:08,677 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:08,678 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:08,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793208336_0001_00_000002, stoppedTime=1452793208678, exitStatus=0
2016-01-14 17:40:08,678 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:08,677 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793208336_0001_00_000002
2016-01-14 17:40:08,678 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:08,678 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793208336_0001_1
2016-01-14 17:40:08,679 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793208336_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:40:08,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:08,679 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793208336_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:08,680 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793208336_0001_00_000002, Exiting
2016-01-14 17:40:08,680 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793208336_0001_00_000002
2016-01-14 17:40:08,680 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793208336_0001_00_000002
2016-01-14 17:40:08,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793208336_0001_1
2016-01-14 17:40:08,681 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2181E041491F4613B8053CD7D894A90F for vertex: vertex_1452793208336_0001_1_01 [FF9E8A6F8FDB40AEBC8EC81D5338669E]
2016-01-14 17:40:08,681 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793208336_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:08,681 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793208336_0001_00_000002 completed successfully
2016-01-14 17:40:08,688 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793208336_0001_1_00 [ECDAAA0E643E459C8EBF72D0ECEA462A]
2016-01-14 17:40:08,690 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793208336_0001_1, queueSize=0
2016-01-14 17:40:08,690 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:08,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208336_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793208336_0001_1, startTime=1452793208503, finishTime=1452793208688, timeTaken=185, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1132462080, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905586417259, Process_Duration=8, Process_End_Time=2905586417267, Read_Duration=2, Tuples_Read=40, Tuples_Written=37, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 17:40:08,690 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793208336_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:08,690 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793208336_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:08,691 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793208336_0001_00_000002
2016-01-14 17:40:08,691 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208336_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:08 Completed Dag: dag_1452793208336_0001_1
2016-01-14 17:40:08,692 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793208336_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:08,692 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793208336_0001_1_post
2016-01-14 17:40:08,693 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:08,693 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:08,698 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount3/count3, applicationId=application_1452793208336_0001
2016-01-14 17:40:08,700 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:08,700 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:08,700 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:08,700 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:08,700 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:08,700 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:08,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:08,701 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:08,701 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:08,701 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:08,702 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:08,702 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:08,702 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:08,706 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2647E7DA3EA24FC8AF5635743B4842A8/E9DDD644A8BE4C929096B3DC01C60494/.tez/application_1452793208336_0001
2016-01-14 17:40:08,706 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.393
2016-01-14 17:40:08,778 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount3/count3
2016-01-14 17:40:08,803 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:08,803 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:08,850 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 17:40:08,852 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-01-14 17:40:08,852 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:08,853 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:08,853 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:08,870 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt
2016-01-14 17:40:08,876 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4
2016-01-14 17:40:08,928 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting
2016-01-14 17:40:08,928 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:08,929 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-01-14 17:40:08,929 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] parallel execution of steps is enabled: false
2016-01-14 17:40:08,929 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] executing total steps: 1
2016-01-14 17:40:08,930 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] allocating management threads: 1
2016-01-14 17:40:08,930 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] starting step: (1/1) ...latform/testCount4/count4
2016-01-14 17:40:08,931 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [count] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:08,944 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] tez session mode enabled: true
2016-01-14 17:40:08,950 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:08,950 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:08,951 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:08,962 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2/.tez/application_1452793208950_0001 doesn't exist and is created
2016-01-14 17:40:08,976 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:08,976 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:08,976 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2/.tez/application_1452793208950_0001
2016-01-14 17:40:08,978 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793208950_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:08,989 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:08,990 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:08,991 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:08,991 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:08,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2/.tez/application_1452793208950_0001/localmode-log-dir
2016-01-14 17:40:08,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2/.tez/application_1452793208950_0001/localmode-log-dir/history.txt.appattempt_1452793208950_0001_000000, maxErrors=10
2016-01-14 17:40:08,992 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:08,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793208950_0001_000000, appSubmitTime=1452793208978, launchTime=1452793208978
2016-01-14 17:40:08,999 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:09,001 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:09,001 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:09,002 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:09,004 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:39304
2016-01-14 17:40:09,005 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:09,007 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793208950_0001_000000, startTime=1452793209007
2016-01-14 17:40:09,007 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:09,007 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:09,076 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:09,076 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:09,076 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1452793208950_0001, dagName=[A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2] count/(1/1) ...latform/testCount4/count4
2016-01-14 17:40:09,094 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2] count/(1/1) ...latform/testCount4/count4
2016-01-14 17:40:09,096 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793208950_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2/.tez/application_1452793208950_0001/localmode-log-dir/dag_1452793208950_0001_1.dot
2016-01-14 17:40:09,096 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2/.tez/application_1452793208950_0001/localmode-log-dir/dag_1452793208950_0001_1-tez-dag.pb.txt
2016-01-14 17:40:09,106 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793208950_0001_1
2016-01-14 17:40:09,106 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:09,106 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2] count/(1/1) ...latform/testCount4/count4
2016-01-14 17:40:09 Running Dag: dag_1452793208950_0001_1
2016-01-14 17:40:09,116 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:09,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793208950_0001_1, submitTime=1452793209094
2016-01-14 17:40:09,117 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DEE45D5161DE45D1AE79BFE68AC3012F
2016-01-14 17:40:09,117 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:09,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:09,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793208950_0001_1, initTime=1452793209116
2016-01-14 17:40:09,118 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793208950_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:09,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:DAG_STARTED]: dagID=dag_1452793208950_0001_1, startTime=1452793209118
2016-01-14 17:40:09,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:09,119 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:09,119 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793208950_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:09,119 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FE3067B1B3ED47A39B5D26454A334D20 : {112B84C15FE14889BC10BE3FC2905B88={InputName=112B84C15FE14889BC10BE3FC2905B88}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:09,119 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1452793208950_0001, dagName=[A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2] count/(1/1) ...latform/testCount4/count4
2016-01-14 17:40:09,119 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 112B84C15FE14889BC10BE3FC2905B88, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:09,120 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20]
2016-01-14 17:40:09,120 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20]
2016-01-14 17:40:09,120 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20]
2016-01-14 17:40:09,120 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20]. Starting root input initializers: 1
2016-01-14 17:40:09,121 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:09,121 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 112B84C15FE14889BC10BE3FC2905B88 on vertex vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20]
2016-01-14 17:40:09,121 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F]
2016-01-14 17:40:09,123 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:09,123 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 112B84C15FE14889BC10BE3FC2905B88 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:09,123 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:09,124 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:09,124 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:40:09,125 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F]
2016-01-14 17:40:09,125 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F]
2016-01-14 17:40:09,125 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F]
2016-01-14 17:40:09,125 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A0636C00D6754542AA71B6CEBF6D620B, vertexId=vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:09,125 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A0636C00D6754542AA71B6CEBF6D620B, vertexId=vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F]
2016-01-14 17:40:09,127 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A0636C00D6754542AA71B6CEBF6D620B, vertexName=DEE45D5161DE45D1AE79BFE68AC3012F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:09,125 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 402
2016-01-14 17:40:09,127 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A0636C00D6754542AA71B6CEBF6D620B, vertexId=vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F]
2016-01-14 17:40:09,128 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 112B84C15FE14889BC10BE3FC2905B88 on vertex vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20]
2016-01-14 17:40:09,128 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:09,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DEE45D5161DE45D1AE79BFE68AC3012F, vertexId=vertex_1452793208950_0001_1_01, initRequestedTime=1452793209121, initedTime=1452793209128, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:40:09,129 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:40:09,129 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:09,129 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {112B84C15FE14889BC10BE3FC2905B88=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:09,130 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20] parallelism set to 1
2016-01-14 17:40:09,130 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:09,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793208950_0001_1_00, eventCount=1
2016-01-14 17:40:09,131 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:09,131 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FE3067B1B3ED47A39B5D26454A334D20, vertexId=vertex_1452793208950_0001_1_00, initRequestedTime=1452793209119, initedTime=1452793209130, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:09,131 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20] with distanceFromRoot: 0
2016-01-14 17:40:09,131 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:09,132 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FE3067B1B3ED47A39B5D26454A334D20
2016-01-14 17:40:09,132 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20]
2016-01-14 17:40:09,132 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:09,133 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793208950_0001_1_00, startRequestedTime=1452793209132, startedTime=1452793209132
2016-01-14 17:40:09,133 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:09,133 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:09,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:TASK_STARTED]: vertexName=FE3067B1B3ED47A39B5D26454A334D20, taskId=task_1452793208950_0001_1_00_000000, scheduledTime=1452793209133, launchTime=1452793209133
2016-01-14 17:40:09,134 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208950_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:09,134 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793208950_0001_1_00 for vertex: vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F] numStartedSources: 1 numSources: 1
2016-01-14 17:40:09,134 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F] with distanceFromRoot: 1
2016-01-14 17:40:09,134 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20]
2016-01-14 17:40:09,135 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FE3067B1B3ED47A39B5D26454A334D20 in vertex: DEE45D5161DE45D1AE79BFE68AC3012F
2016-01-14 17:40:09,135 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DEE45D5161DE45D1AE79BFE68AC3012F with 1 source tasks and 1 pending tasks
2016-01-14 17:40:09,135 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:09,136 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793208950_0001_1_01, startRequestedTime=1452793209135, startedTime=1452793209135
2016-01-14 17:40:09,136 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:09,136 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2] count/(1/1) ...latform/testCount4/count4, VertexName: FE3067B1B3ED47A39B5D26454A334D20, VertexParallelism: 1, TaskAttemptID:attempt_1452793208950_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=112B84C15FE14889BC10BE3FC2905B88, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DEE45D5161DE45D1AE79BFE68AC3012F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:09,136 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208950_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:09,137 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:09,137 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:09,138 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:09,138 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:09,138 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793208950_0001_00_000001
2016-01-14 17:40:09,138 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:09,139 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] submitted tez dag to app master: application_1452793208950_0001, with dag id: dag_1452793208950_0001_1
2016-01-14 17:40:09,141 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793208950_0001_00_000001
2016-01-14 17:40:09,141 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:09,142 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:09,142 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:09,142 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793208950_0001_00_000001, launchTime=1452793209142
2016-01-14 17:40:09,143 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793208950_0001_00_000001
2016-01-14 17:40:09,144 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793208950_0001_1_00_000000_0] to container: [container_1452793208950_0001_00_000001]
2016-01-14 17:40:09,144 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:09,145 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793208950_0001_1_00_000000_0] started. Is using containerId: [container_1452793208950_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:09,145 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FE3067B1B3ED47A39B5D26454A334D20, taskAttemptId=attempt_1452793208950_0001_1_00_000000_0, startTime=1452793209144, containerId=container_1452793208950_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793208950_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:09,145 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208950_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:09,145 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208950_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:09,145 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:09,146 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793208950_0001_00_000001 given task: attempt_1452793208950_0001_1_00_000000_0
2016-01-14 17:40:09,146 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793208950_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793208950_0001_1_00_000000_0
2016-01-14 17:40:09 Starting to run new task attempt: attempt_1452793208950_0001_1_00_000000_0
2016-01-14 17:40:09,147 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793208950_0001_1_00_000000_0
2016-01-14 17:40:09,147 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:09,147 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:09,147 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:09,147 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:09,147 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:09,147 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:09,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2] count/(1/1) ...latform/testCount4/count4, VertexName: FE3067B1B3ED47A39B5D26454A334D20, VertexParallelism: 1, TaskAttemptID:attempt_1452793208950_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=112B84C15FE14889BC10BE3FC2905B88, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DEE45D5161DE45D1AE79BFE68AC3012F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:09,148 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:09,148 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793208950_0001_1_00_000000_0
2016-01-14 17:40:09,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:09,148 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:09,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=112B84C15FE14889BC10BE3FC2905B88, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:09,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:09,149 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:09,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 112B84C15FE14889BC10BE3FC2905B88
2016-01-14 17:40:09,150 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793208950_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:09,151 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 112B84C15FE14889BC10BE3FC2905B88
2016-01-14 17:40:09,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DEE45D5161DE45D1AE79BFE68AC3012F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:09,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:09,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DEE45D5161DE45D1AE79BFE68AC3012F
2016-01-14 17:40:09,169 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:09,169 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DEE45D5161DE45D1AE79BFE68AC3012F
2016-01-14 17:40:09,171 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:09,171 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:09,172 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 112B84C15FE14889BC10BE3FC2905B88
2016-01-14 17:40:09,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 112B84C15FE14889BC10BE3FC2905B88
2016-01-14 17:40:09,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DEE45D5161DE45D1AE79BFE68AC3012F
2016-01-14 17:40:09,175 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FE3067B1B3ED47A39B5D26454A334D20, ordinal: 0
2016-01-14 17:40:09,176 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE3067B1B3ED47A39B5D26454A334D20, mem on start (mb), free: 506, total: 541, max: 672
2016-01-14 17:40:09,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:09,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:09,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:09,176 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:09,176 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:09,176 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:09,177 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:09,177 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:09,177 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:09,177 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:09,177 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 112B84C15FE14889BC10BE3FC2905B88, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:09,177 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DEE45D5161DE45D1AE79BFE68AC3012F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:09,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:09,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 112B84C15FE14889BC10BE3FC2905B88
2016-01-14 17:40:09,178 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:09,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 112B84C15FE14889BC10BE3FC2905B88
2016-01-14 17:40:09,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 112B84C15FE14889BC10BE3FC2905B88 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:09,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:09,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:09,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:09,180 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793208950_0001_1_00_000000_0
2016-01-14 17:40:09,180 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:09,181 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@71ccb91b
2016-01-14 17:40:09,182 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@71ccb91b
2016-01-14 17:40:09,182 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:09,182 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:09,183 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:09,184 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"] streamed: true, id: 112B84C15FE14889BC10BE3FC2905B88
2016-01-14 17:40:09,184 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(count)[by:[{1}:1]], id: DD80BC430233499DACA3A6679BCD4816
2016-01-14 17:40:09,184 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(count)[by:[{1}:1]] DD80BC430233499DACA3A6679BCD4816
2016-01-14 17:40:09,185 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@5e6c65be; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1e8bcd60; comparator=cascading.tuple.hadoop.util.TupleComparator@6c58694a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:09,186 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:09,186 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:09,186 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:09,199 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:09,199 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:09,199 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:09,199 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:09,199 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:09,201 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/ips.20.txt"]
2016-01-14 17:40:09,201 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:09,202 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FE3067B1B3ED47A39B5D26454A334D20, all 1 inputs ready in: 00:00:00.001
2016-01-14 17:40:09,204 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793208950_0001_1_00_000000_0
2016-01-14 17:40:09,204 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE3067B1B3ED47A39B5D26454A334D20, mem on close (mb), free: 404, total: 541, max: 672
2016-01-14 17:40:09,205 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:09,205 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:09,205 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 466; bufvoid = 104857600
2016-01-14 17:40:09,205 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 17:40:09,208 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:09 Completed running task attempt: attempt_1452793208950_0001_1_00_000000_0
2016-01-14 17:40:09,212 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793208950_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:09,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793208950_0001_1_00, eventCount=1
2016-01-14 17:40:09,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=16, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793209202, Process_Duration=2, Process_End_Time=1452793209204, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 17:40:09,216 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:09,216 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:09,216 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:09,216 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FE3067B1B3ED47A39B5D26454A334D20, taskAttemptId=attempt_1452793208950_0001_1_00_000000_0, startTime=1452793209144, finishTime=1452793209216, timeTaken=72, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=16, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793209202, Process_Duration=2, Process_End_Time=1452793209204, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:40:09,216 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208950_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:09,217 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793208950_0001_00_000001
2016-01-14 17:40:09,224 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:09,224 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793208950_0001_1_00_000000_0
2016-01-14 17:40:09,225 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:09,225 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:TASK_FINISHED]: vertexName=FE3067B1B3ED47A39B5D26454A334D20, taskId=task_1452793208950_0001_1_00_000000, startTime=1452793209144, finishTime=1452793209224, timeTaken=80, status=SUCCEEDED, successfulAttemptID=attempt_1452793208950_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=16, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793209202, Process_Duration=2, Process_End_Time=1452793209204, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 17:40:09,225 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208950_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:09,225 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20] : 1
2016-01-14 17:40:09,225 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20]
2016-01-14 17:40:09,226 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:09,227 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:VERTEX_FINISHED]: vertexName=FE3067B1B3ED47A39B5D26454A334D20, vertexId=vertex_1452793208950_0001_1_00, initRequestedTime=1452793209119, initedTime=1452793209130, startRequestedTime=1452793209132, startedTime=1452793209132, finishTime=1452793209226, timeTaken=94, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=445, FILE_BYTES_WRITTEN=495, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=16, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793209202, Process_Duration=2, Process_End_Time=1452793209204, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452793209144, firstTasksToStart=[ task_1452793208950_0001_1_00_000000 ], lastTaskFinishTime=1452793209216, lastTasksToFinish=[ task_1452793208950_0001_1_00_000000 ], minTaskDuration=72, maxTaskDuration=72, avgTaskDuration=72.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793208950_0001_1_00_000000 ], longestDurationTasks=[ task_1452793208950_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:09,227 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:09,227 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:09,227 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F] attempt: attempt_1452793208950_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:09,227 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DEE45D5161DE45D1AE79BFE68AC3012F
2016-01-14 17:40:09,228 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F]
2016-01-14 17:40:09,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793208950_0001_00_000001, stoppedTime=1452793209228, exitStatus=0
2016-01-14 17:40:09,228 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:09,228 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:09,228 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:09,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:TASK_STARTED]: vertexName=DEE45D5161DE45D1AE79BFE68AC3012F, taskId=task_1452793208950_0001_1_01_000000, scheduledTime=1452793209228, launchTime=1452793209228
2016-01-14 17:40:09,228 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:09,229 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208950_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:09,230 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793208950_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:40:09,230 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F]
2016-01-14 17:40:09,230 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:09,230 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2] count/(1/1) ...latform/testCount4/count4, VertexName: DEE45D5161DE45D1AE79BFE68AC3012F, VertexParallelism: 1, TaskAttemptID:attempt_1452793208950_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FE3067B1B3ED47A39B5D26454A334D20, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A0636C00D6754542AA71B6CEBF6D620B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:09,231 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208950_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:09,230 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793208950_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:09,231 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:09,231 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793208950_0001_00_000001: 14 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:09,232 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793208950_0001_00_000002
2016-01-14 17:40:09,232 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793208950_0001_00_000001, Exiting
2016-01-14 17:40:09,232 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793208950_0001_00_000001
2016-01-14 17:40:09,232 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793208950_0001_00_000001
2016-01-14 17:40:09,233 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:09,235 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793208950_0001_00_000002
2016-01-14 17:40:09,236 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 17:40:09,237 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793208950_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:09,238 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793208950_0001_00_000001 completed successfully
2016-01-14 17:40:09,238 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:09,241 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:09,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:09,242 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793208950_0001_00_000002, launchTime=1452793209241
2016-01-14 17:40:09,242 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793208950_0001_00_000002
2016-01-14 17:40:09,243 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793208950_0001_1_01_000000_0] to container: [container_1452793208950_0001_00_000002]
2016-01-14 17:40:09,243 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:09,244 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793208950_0001_1_01_000000_0] started. Is using containerId: [container_1452793208950_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:40:09,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DEE45D5161DE45D1AE79BFE68AC3012F, taskAttemptId=attempt_1452793208950_0001_1_01_000000_0, startTime=1452793209244, containerId=container_1452793208950_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793208950_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:40:09,245 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208950_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:09,245 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208950_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:09,245 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:09,246 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793208950_0001_00_000002 given task: attempt_1452793208950_0001_1_01_000000_0
2016-01-14 17:40:09,246 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793208950_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793208950_0001_1_01_000000_0
2016-01-14 17:40:09 Starting to run new task attempt: attempt_1452793208950_0001_1_01_000000_0
2016-01-14 17:40:09,247 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793208950_0001_1_01_000000_0
2016-01-14 17:40:09,247 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:09,247 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:09,247 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:09,247 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:09,247 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:09,247 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:09,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2] count/(1/1) ...latform/testCount4/count4, VertexName: DEE45D5161DE45D1AE79BFE68AC3012F, VertexParallelism: 1, TaskAttemptID:attempt_1452793208950_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FE3067B1B3ED47A39B5D26454A334D20, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A0636C00D6754542AA71B6CEBF6D620B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:09,248 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:09,248 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793208950_0001_1_01_000000_0
2016-01-14 17:40:09,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:09,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:09,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FE3067B1B3ED47A39B5D26454A334D20, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:09,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:09,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FE3067B1B3ED47A39B5D26454A334D20
2016-01-14 17:40:09,252 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:09,252 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:09,252 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FE3067B1B3ED47A39B5D26454A334D20
2016-01-14 17:40:09,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FE3067B1B3ED47A39B5D26454A334D20
2016-01-14 17:40:09,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:09,253 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793208950_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:40:09,254 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A0636C00D6754542AA71B6CEBF6D620B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:09,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:09,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A0636C00D6754542AA71B6CEBF6D620B
2016-01-14 17:40:09,255 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:09,256 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A0636C00D6754542AA71B6CEBF6D620B
2016-01-14 17:40:09,256 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:09,264 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DEE45D5161DE45D1AE79BFE68AC3012F, ordinal: 1
2016-01-14 17:40:09,265 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DEE45D5161DE45D1AE79BFE68AC3012F, mem on start (mb), free: 398, total: 541, max: 672
2016-01-14 17:40:09,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:09,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:09,265 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:09,267 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:09,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A0636C00D6754542AA71B6CEBF6D620B
2016-01-14 17:40:09,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:09,268 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:09,268 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:09,268 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:09,269 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:09,269 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:09,269 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:40:09,269 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FE3067B1B3ED47A39B5D26454A334D20, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:40:09,270 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A0636C00D6754542AA71B6CEBF6D620B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:09,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:09,270 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FE3067B1B3ED47A39B5D26454A334D20
2016-01-14 17:40:09,271 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:09,271 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:09,271 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FE3067B1B3ED47A39B5D26454A334D20 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:09,271 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:40:09,271 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:09,271 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:09,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FE3067B1B3ED47A39B5D26454A334D20 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:09,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FE3067B1B3ED47A39B5D26454A334D20
2016-01-14 17:40:09,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:09,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:09,273 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:09,278 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793208950_0001_1_01_000000_0
2016-01-14 17:40:09,279 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793208950_0001_1_00_000000_0_10080, runDuration: 40843, ]
2016-01-14 17:40:09,281 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(count)[by:[{1}:1]]
2016-01-14 17:40:09,281 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793208950_0001_1_00_000000_0_10080] to DISK_DIRECT, CompressedSize=463, DecompressedSize=459,EndTime=1452793209281, TimeTaken=1, Rate=0.44 MB/s
2016-01-14 17:40:09,282 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FE3067B1B3ED47A39B5D26454A334D20
2016-01-14 17:40:09,282 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(count)[by:[{1}:1]] streamed: true, id: DD80BC430233499DACA3A6679BCD4816
2016-01-14 17:40:09,282 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:09,282 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"], id: A0636C00D6754542AA71B6CEBF6D620B
2016-01-14 17:40:09,284 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testCount4/count4"]
2016-01-14 17:40:09,284 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(count)[by:[{1}:1]] DD80BC430233499DACA3A6679BCD4816, for 1 inputs
2016-01-14 17:40:09,284 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FE3067B1B3ED47A39B5D26454A334D20] #13 in 5ms
2016-01-14 17:40:09,284 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FE3067B1B3ED47A39B5D26454A334D20] #13, status:false, isInterrupted:false
2016-01-14 17:40:09,287 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:09,287 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 463 bytes from disk
2016-01-14 17:40:09,287 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:09,287 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:09,287 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 434 bytes
2016-01-14 17:40:09,288 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FE3067B1B3ED47A39B5D26454A334D20
2016-01-14 17:40:09,288 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:09,288 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DEE45D5161DE45D1AE79BFE68AC3012F, all 1 inputs ready in: 00:00:00.004
2016-01-14 17:40:09,290 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6cd83a0c; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@67cb21a7
2016-01-14 17:40:09,294 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:09,294 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:09,294 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793208950_0001_1_01_000000_0
2016-01-14 17:40:09,295 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793208950_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:40:09,295 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793208950_0001_1_01_000000_0
2016-01-14 17:40:09,295 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DEE45D5161DE45D1AE79BFE68AC3012F, mem on close (mb), free: 396, total: 541, max: 672
2016-01-14 17:40:09,296 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FE3067B1B3ED47A39B5D26454A334D20
2016-01-14 17:40:09,296 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:09,296 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:09,296 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:09,297 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:09,297 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:09 Completed running task attempt: attempt_1452793208950_0001_1_01_000000_0
2016-01-14 17:40:09,298 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793208950_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:09,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793209289, Process_Duration=6, Process_End_Time=1452793209295, Read_Duration=3, Tuples_Read=20, Tuples_Written=17, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=17]]
2016-01-14 17:40:09,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:09,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DEE45D5161DE45D1AE79BFE68AC3012F, taskAttemptId=attempt_1452793208950_0001_1_01_000000_0, startTime=1452793209244, finishTime=1452793209299, timeTaken=55, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793209289, Process_Duration=6, Process_End_Time=1452793209295, Read_Duration=3, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 17:40:09,300 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793208950_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:09,300 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793208950_0001_1_01_000000_0
2016-01-14 17:40:09,300 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:09,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:09,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:TASK_FINISHED]: vertexName=DEE45D5161DE45D1AE79BFE68AC3012F, taskId=task_1452793208950_0001_1_01_000000, startTime=1452793209244, finishTime=1452793209300, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1452793208950_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793209289, Process_Duration=6, Process_End_Time=1452793209295, Read_Duration=3, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17
2016-01-14 17:40:09,301 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793208950_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:09,300 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:09,301 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:09,302 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F] : 1
2016-01-14 17:40:09,302 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F]
2016-01-14 17:40:09,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:VERTEX_FINISHED]: vertexName=DEE45D5161DE45D1AE79BFE68AC3012F, vertexId=vertex_1452793208950_0001_1_01, initRequestedTime=1452793209121, initedTime=1452793209128, startRequestedTime=1452793209135, startedTime=1452793209135, finishTime=1452793209302, timeTaken=167, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=495, FILE_BYTES_WRITTEN=282, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, OUTPUT_RECORDS=17, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793209289, Process_Duration=6, Process_End_Time=1452793209295, Read_Duration=3, Tuples_Read=20, Tuples_Written=17, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=17, vertexStats=firstTaskStartTime=1452793209244, firstTasksToStart=[ task_1452793208950_0001_1_01_000000 ], lastTaskFinishTime=1452793209299, lastTasksToFinish=[ task_1452793208950_0001_1_01_000000 ], minTaskDuration=55, maxTaskDuration=55, avgTaskDuration=55.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793208950_0001_1_01_000000 ], longestDurationTasks=[ task_1452793208950_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:09,303 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:09,303 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:09,304 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793208950_0001_00_000002, stoppedTime=1452793209304, exitStatus=0
2016-01-14 17:40:09,304 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:09,304 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:09,304 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793208950_0001_1
2016-01-14 17:40:09,304 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793208950_0001_1
2016-01-14 17:40:09,304 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A0636C00D6754542AA71B6CEBF6D620B for vertex: vertex_1452793208950_0001_1_01 [DEE45D5161DE45D1AE79BFE68AC3012F]
2016-01-14 17:40:09,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:09,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:09,302 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793208950_0001_00_000002
2016-01-14 17:40:09,306 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793208950_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:40:09,306 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793208950_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:09,307 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793208950_0001_00_000002, Exiting
2016-01-14 17:40:09,307 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793208950_0001_00_000002
2016-01-14 17:40:09,307 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793208950_0001_00_000002
2016-01-14 17:40:09,307 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793208950_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:09,307 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793208950_0001_00_000002 completed successfully
2016-01-14 17:40:09,313 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793208950_0001_1_00 [FE3067B1B3ED47A39B5D26454A334D20]
2016-01-14 17:40:09,316 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793208950_0001_1, queueSize=0
2016-01-14 17:40:09,317 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:09,318 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793208950_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793208950_0001_1, startTime=1452793209118, finishTime=1452793209314, timeTaken=196, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=940, FILE_BYTES_WRITTEN=777, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=16, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=16, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1135607808, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=37, OUTPUT_BYTES=466, OUTPUT_BYTES_WITH_OVERHEAD=459, OUTPUT_BYTES_PHYSICAL=463, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=463, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=463, SHUFFLE_BYTES_DECOMPRESSED=459, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=463, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=17, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905586418491, Process_Duration=8, Process_End_Time=2905586418499, Read_Duration=4, Tuples_Read=40, Tuples_Written=37, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=17
2016-01-14 17:40:09,318 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793208950_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:09,318 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793208950_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:09,319 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793208950_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:09,319 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793208950_0001_00_000002
2016-01-14 17:40:09 Completed Dag: dag_1452793208950_0001_1
2016-01-14 17:40:09,320 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793208950_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:09,321 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793208950_0001_1_post
2016-01-14 17:40:09,321 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:09,322 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:09,326 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testCount4/count4, applicationId=application_1452793208950_0001
2016-01-14 17:40:09,328 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:09,328 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:09,328 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:09,328 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:09,328 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:09,329 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:09,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:09,329 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:09,330 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:09,330 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:09,330 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:09,330 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:09,331 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:09,336 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/A987CD9F1AD5419FB280D4702DC50213/CA982F984DED40A5BC4F260441F23BA2/.tez/application_1452793208950_0001
2016-01-14 17:40:09,336 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [count] completed in: 00:00.408
2016-01-14 17:40:09,355 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:40:09,355 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:40:09,410 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testCount4/count4
2016-01-14 17:40:09,433 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:09,433 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:09,466 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 17:40:09,472 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 17:40:09,473 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 17:40:09,473 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 1
2016-01-14 17:40:09,473 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:09,488 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 17:40:09,489 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter
2016-01-14 17:40:09,498 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 17:40:09,498 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:09,498 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-01-14 17:40:09,499 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 17:40:09,499 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 17:40:09,499 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 17:40:09,500 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...latform/testFilter/filter
2016-01-14 17:40:09,500 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:09,514 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 17:40:09,518 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:09,519 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:09,519 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:09,527 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878/.tez/application_1452793209518_0001 doesn't exist and is created
2016-01-14 17:40:09,541 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:09,541 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:09,541 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878/.tez/application_1452793209518_0001
2016-01-14 17:40:09,544 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793209518_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:09,555 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:09,556 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:09,557 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:09,557 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:09,558 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878/.tez/application_1452793209518_0001/localmode-log-dir
2016-01-14 17:40:09,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878/.tez/application_1452793209518_0001/localmode-log-dir/history.txt.appattempt_1452793209518_0001_000000, maxErrors=10
2016-01-14 17:40:09,559 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:09,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793209518_0001_000000, appSubmitTime=1452793209543, launchTime=1452793209543
2016-01-14 17:40:09,562 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:09,564 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:09,565 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:09,565 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:09,571 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:40558
2016-01-14 17:40:09,572 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:09,574 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793209518_0001_000000, startTime=1452793209574
2016-01-14 17:40:09,574 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:09,574 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:09,641 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:09,641 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:09,641 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1452793209518_0001, dagName=[55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878] test/(1/1) ...latform/testFilter/filter
2016-01-14 17:40:09,657 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878] test/(1/1) ...latform/testFilter/filter
2016-01-14 17:40:09,659 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793209518_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878/.tez/application_1452793209518_0001/localmode-log-dir/dag_1452793209518_0001_1.dot
2016-01-14 17:40:09,659 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878/.tez/application_1452793209518_0001/localmode-log-dir/dag_1452793209518_0001_1-tez-dag.pb.txt
2016-01-14 17:40:09,671 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793209518_0001_1
2016-01-14 17:40:09,671 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:09,671 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878] test/(1/1) ...latform/testFilter/filter
2016-01-14 17:40:09 Running Dag: dag_1452793209518_0001_1
2016-01-14 17:40:09,680 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:09,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793209518_0001_1, submitTime=1452793209657
2016-01-14 17:40:09,681 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex ADDE86DA48794017B9483D5E41C978BB
2016-01-14 17:40:09,681 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:09,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:09,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793209518_0001_1, initTime=1452793209680
2016-01-14 17:40:09,682 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793209518_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:09,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:DAG_STARTED]: dagID=dag_1452793209518_0001_1, startTime=1452793209682
2016-01-14 17:40:09,682 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:09,682 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793209518_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:09,683 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: ADDE86DA48794017B9483D5E41C978BB : {6F932CFF5A5C4651942E1CC7C219847F={InputName=6F932CFF5A5C4651942E1CC7C219847F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:09,683 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1452793209518_0001, dagName=[55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878] test/(1/1) ...latform/testFilter/filter
2016-01-14 17:40:09,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:09,683 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6F932CFF5A5C4651942E1CC7C219847F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:09,683 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,683 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,684 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,684 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]. Starting root input initializers: 1
2016-01-14 17:40:09,684 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:09,685 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6F932CFF5A5C4651942E1CC7C219847F on vertex vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,687 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:09,687 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6F932CFF5A5C4651942E1CC7C219847F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:09,688 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:09,688 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:09,690 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 17:40:09,691 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6F932CFF5A5C4651942E1CC7C219847F on vertex vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,691 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:09,691 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6F932CFF5A5C4651942E1CC7C219847F=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:09,691 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB] parallelism set to 1
2016-01-14 17:40:09,691 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:09,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793209518_0001_1_00, eventCount=1
2016-01-14 17:40:09,692 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,692 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9C61040ED1A8480A9ED4D26416B5AEC8, vertexId=vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:09,692 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9C61040ED1A8480A9ED4D26416B5AEC8, vertexId=vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,694 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9C61040ED1A8480A9ED4D26416B5AEC8, vertexName=ADDE86DA48794017B9483D5E41C978BB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:09,694 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9C61040ED1A8480A9ED4D26416B5AEC8, vertexId=vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,695 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:09,695 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:VERTEX_INITIALIZED]: vertexName=ADDE86DA48794017B9483D5E41C978BB, vertexId=vertex_1452793209518_0001_1_00, initRequestedTime=1452793209683, initedTime=1452793209695, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:09,695 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB] with distanceFromRoot: 0
2016-01-14 17:40:09,695 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:09,695 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in ADDE86DA48794017B9483D5E41C978BB
2016-01-14 17:40:09,696 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,696 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:09,696 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793209518_0001_1_00, startRequestedTime=1452793209695, startedTime=1452793209695
2016-01-14 17:40:09,697 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:09,697 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:09,697 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:TASK_STARTED]: vertexName=ADDE86DA48794017B9483D5E41C978BB, taskId=task_1452793209518_0001_1_00_000000, scheduledTime=1452793209697, launchTime=1452793209697
2016-01-14 17:40:09,697 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793209518_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:09,698 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,698 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878] test/(1/1) ...latform/testFilter/filter, VertexName: ADDE86DA48794017B9483D5E41C978BB, VertexParallelism: 1, TaskAttemptID:attempt_1452793209518_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6F932CFF5A5C4651942E1CC7C219847F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9C61040ED1A8480A9ED4D26416B5AEC8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:09,698 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793209518_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:09,699 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:09,699 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:09,700 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:09,700 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:09,701 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793209518_0001_00_000001
2016-01-14 17:40:09,701 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452793209518_0001, with dag id: dag_1452793209518_0001_1
2016-01-14 17:40:09,701 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209518_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:09,703 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793209518_0001_00_000001
2016-01-14 17:40:09,703 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:09,703 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:09,704 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209518_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:09,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793209518_0001_00_000001, launchTime=1452793209703
2016-01-14 17:40:09,705 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793209518_0001_00_000001
2016-01-14 17:40:09,706 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793209518_0001_1_00_000000_0] to container: [container_1452793209518_0001_00_000001]
2016-01-14 17:40:09,706 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209518_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:09,706 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793209518_0001_00_000001 given task: attempt_1452793209518_0001_1_00_000000_0
2016-01-14 17:40:09,707 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793209518_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793209518_0001_1_00_000000_0
2016-01-14 17:40:09,707 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793209518_0001_1_00_000000_0] started. Is using containerId: [container_1452793209518_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:09,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:09,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=ADDE86DA48794017B9483D5E41C978BB, taskAttemptId=attempt_1452793209518_0001_1_00_000000_0, startTime=1452793209707, containerId=container_1452793209518_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793209518_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:09,707 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793209518_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:09 Starting to run new task attempt: attempt_1452793209518_0001_1_00_000000_0
2016-01-14 17:40:09,708 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793209518_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:09,708 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793209518_0001_1_00_000000_0
2016-01-14 17:40:09,708 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:09,708 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:09,708 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:09,709 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:09,709 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:09,709 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:09,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878] test/(1/1) ...latform/testFilter/filter, VertexName: ADDE86DA48794017B9483D5E41C978BB, VertexParallelism: 1, TaskAttemptID:attempt_1452793209518_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6F932CFF5A5C4651942E1CC7C219847F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=9C61040ED1A8480A9ED4D26416B5AEC8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:09,710 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:09,710 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793209518_0001_1_00_000000_0
2016-01-14 17:40:09,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:09,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:09,711 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793209518_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:09,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9C61040ED1A8480A9ED4D26416B5AEC8, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:09,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:09,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9C61040ED1A8480A9ED4D26416B5AEC8
2016-01-14 17:40:09,712 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:09,712 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9C61040ED1A8480A9ED4D26416B5AEC8
2016-01-14 17:40:09,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:09,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6F932CFF5A5C4651942E1CC7C219847F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:09,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:09,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6F932CFF5A5C4651942E1CC7C219847F
2016-01-14 17:40:09,713 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6F932CFF5A5C4651942E1CC7C219847F
2016-01-14 17:40:09,714 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:09,715 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:09,715 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6F932CFF5A5C4651942E1CC7C219847F
2016-01-14 17:40:09,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6F932CFF5A5C4651942E1CC7C219847F
2016-01-14 17:40:09,722 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: ADDE86DA48794017B9483D5E41C978BB, ordinal: 0
2016-01-14 17:40:09,722 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADDE86DA48794017B9483D5E41C978BB, mem on start (mb), free: 311, total: 541, max: 672
2016-01-14 17:40:09,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:09,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:09,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:09,726 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:09,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9C61040ED1A8480A9ED4D26416B5AEC8
2016-01-14 17:40:09,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:09,726 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:09,727 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:09,727 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:09,727 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:09,727 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:09,727 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:09,727 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 2, TotalRequested: 0, TotalRequestedScaled: 0.0, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.00
2016-01-14 17:40:09,728 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9C61040ED1A8480A9ED4D26416B5AEC8, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:09,728 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6F932CFF5A5C4651942E1CC7C219847F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:09,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:09,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6F932CFF5A5C4651942E1CC7C219847F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:09,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6F932CFF5A5C4651942E1CC7C219847F
2016-01-14 17:40:09,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:09,728 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:09,728 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:09,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6F932CFF5A5C4651942E1CC7C219847F
2016-01-14 17:40:09,729 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:09,729 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793209518_0001_1_00_000000_0
2016-01-14 17:40:09,730 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:09,730 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@53830ac9
2016-01-14 17:40:09,731 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@53830ac9
2016-01-14 17:40:09,731 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:09,731 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:09,733 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:09,734 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 6F932CFF5A5C4651942E1CC7C219847F
2016-01-14 17:40:09,734 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"], id: 9C61040ED1A8480A9ED4D26416B5AEC8
2016-01-14 17:40:09,734 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testFilter/filter"]
2016-01-14 17:40:09,734 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:09,734 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:09,735 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: ADDE86DA48794017B9483D5E41C978BB, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:09,736 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:09,737 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:09,737 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793209518_0001_1_00_000000_0
2016-01-14 17:40:09,737 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793209518_0001_1_00_000000_0 given a go for committing the task output.
2016-01-14 17:40:09,738 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793209518_0001_1_00_000000_0
2016-01-14 17:40:09,738 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: ADDE86DA48794017B9483D5E41C978BB, mem on close (mb), free: 309, total: 541, max: 672
2016-01-14 17:40:09 Completed running task attempt: attempt_1452793209518_0001_1_00_000000_0
2016-01-14 17:40:09,739 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793209518_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:09,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:09,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=ADDE86DA48794017B9483D5E41C978BB, taskAttemptId=attempt_1452793209518_0001_1_00_000000_0, startTime=1452793209707, finishTime=1452793209739, timeTaken=32, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452793209735, Process_Duration=3, Process_End_Time=1452793209738, Read_Duration=0, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-01-14 17:40:09,740 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793209518_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:09,740 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:09,740 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793209518_0001_1_00_000000_0
2016-01-14 17:40:09,741 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:09,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:TASK_FINISHED]: vertexName=ADDE86DA48794017B9483D5E41C978BB, taskId=task_1452793209518_0001_1_00_000000, startTime=1452793209707, finishTime=1452793209741, timeTaken=34, status=SUCCEEDED, successfulAttemptID=attempt_1452793209518_0001_1_00_000000_0, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452793209735, Process_Duration=3, Process_End_Time=1452793209738, Read_Duration=0, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-01-14 17:40:09,741 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793209518_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:09,741 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB] : 1
2016-01-14 17:40:09,741 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 22 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3][cascading.flow.SliceCounters Process_Begin_Time=1452793209735, Process_Duration=3, Process_End_Time=1452793209738, Read_Duration=0, Tuples_Read=10, Tuples_Written=3, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10, Tuples_Written=3]]
2016-01-14 17:40:09,742 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:09,742 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:09,743 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:09,743 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:VERTEX_FINISHED]: vertexName=ADDE86DA48794017B9483D5E41C978BB, vertexId=vertex_1452793209518_0001_1_00, initRequestedTime=1452793209683, initedTime=1452793209695, startRequestedTime=1452793209695, startedTime=1452793209695, finishTime=1452793209742, timeTaken=47, status=SUCCEEDED, diagnostics=, counters=Counters: 22, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452793209735, Process_Duration=3, Process_End_Time=1452793209738, Read_Duration=0, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3, vertexStats=firstTaskStartTime=1452793209707, firstTasksToStart=[ task_1452793209518_0001_1_00_000000 ], lastTaskFinishTime=1452793209739, lastTasksToFinish=[ task_1452793209518_0001_1_00_000000 ], minTaskDuration=32, maxTaskDuration=32, avgTaskDuration=32.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793209518_0001_1_00_000000 ], longestDurationTasks=[ task_1452793209518_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:09,743 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:09,743 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209518_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:09,743 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793209518_0001_00_000001, stoppedTime=1452793209743, exitStatus=0
2016-01-14 17:40:09,743 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209518_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:09,744 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=1
2016-01-14 17:40:09,744 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793209518_0001_1
2016-01-14 17:40:09,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793209518_0001_1
2016-01-14 17:40:09,744 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 9C61040ED1A8480A9ED4D26416B5AEC8 for vertex: vertex_1452793209518_0001_1_00 [ADDE86DA48794017B9483D5E41C978BB]
2016-01-14 17:40:09,744 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793209518_0001_00_000001
2016-01-14 17:40:09,745 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793209518_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:09,745 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793209518_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:09,745 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793209518_0001_00_000001, Exiting
2016-01-14 17:40:09,745 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793209518_0001_00_000001
2016-01-14 17:40:09,745 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793209518_0001_00_000001
2016-01-14 17:40:09,747 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:09,747 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793209518_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:09,748 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793209518_0001_00_000001 completed successfully
2016-01-14 17:40:09,754 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793209518_0001_1, queueSize=0
2016-01-14 17:40:09,754 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:09,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209518_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793209518_0001_1, startTime=1452793209682, finishTime=1452793209752, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 24, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=1, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2174, FILE_BYTES_WRITTEN=507, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=567803904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=3, cascading.flow.SliceCounters, Process_Begin_Time=1452793209735, Process_Duration=3, Process_End_Time=1452793209738, Read_Duration=0, Tuples_Read=10, Tuples_Written=3, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=3
2016-01-14 17:40:09,755 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793209518_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:09,756 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793209518_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:09,756 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793209518_0001_00_000001
2016-01-14 17:40:09,756 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209518_0001_00_000001 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:09 Completed Dag: dag_1452793209518_0001_1
2016-01-14 17:40:09,758 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793209518_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:09,758 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793209518_0001_1_post
2016-01-14 17:40:09,758 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:09,758 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:09,762 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testFilter/filter, applicationId=application_1452793209518_0001
2016-01-14 17:40:09,763 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:09,764 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:09,764 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:09,764 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:09,764 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:09,764 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:09,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:09,765 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:09,765 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:09,765 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:09,765 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:09,765 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:09,765 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:09,771 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/55AD9AB14DBF4619827AE55E142FC83B/C67054F7665947F19D9F27C23592F878/.tez/application_1452793209518_0001
2016-01-14 17:40:09,771 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.273
2016-01-14 17:40:09,817 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testFilter/filter
2016-01-14 17:40:09,845 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 17:40:09,845 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 17:40:09,887 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 17:40:09,892 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.021
2016-01-14 17:40:09,892 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:09,893 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 17:40:09,893 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 17:40:09,909 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt
2016-01-14 17:40:09,916 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple
2016-01-14 17:40:09,966 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting
2016-01-14 17:40:09,967 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:09,967 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] sink: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-01-14 17:40:09,967 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] parallel execution of steps is enabled: false
2016-01-14 17:40:09,968 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] executing total steps: 1
2016-01-14 17:40:09,969 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] allocating management threads: 1
2016-01-14 17:40:09,969 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] starting step: (1/1) ...latform/testSimple/simple
2016-01-14 17:40:09,969 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [test] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 17:40:09,982 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] tez session mode enabled: true
2016-01-14 17:40:09,986 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:09,987 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 17:40:09,987 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 17:40:09,995 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001/.tez/application_1452793209987_0001 doesn't exist and is created
2016-01-14 17:40:10,011 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 17:40:10,011 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 17:40:10,012 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001/.tez/application_1452793209987_0001
2016-01-14 17:40:10,014 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452793209987_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 17:40:10,025 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 17:40:10,026 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 17:40:10,027 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 17:40:10,028 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 17:40:10,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001/.tez/application_1452793209987_0001/localmode-log-dir
2016-01-14 17:40:10,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001/.tez/application_1452793209987_0001/localmode-log-dir/history.txt.appattempt_1452793209987_0001_000000, maxErrors=10
2016-01-14 17:40:10,029 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 17:40:10,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452793209987_0001_000000, appSubmitTime=1452793210013, launchTime=1452793210013
2016-01-14 17:40:10,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 17:40:10,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 17:40:10,034 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 17:40:10,035 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 17:40:10,041 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59469
2016-01-14 17:40:10,043 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 17:40:10,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452793209987_0001_000000, startTime=1452793210045
2016-01-14 17:40:10,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 17:40:10,045 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 17:40:10,111 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 17:40:10,111 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 17:40:10,112 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1452793209987_0001, dagName=[6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001] test/(1/1) ...latform/testSimple/simple
2016-01-14 17:40:10,121 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 17:40:10,122 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 17:40:10,129 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001] test/(1/1) ...latform/testSimple/simple
2016-01-14 17:40:10,132 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452793209987_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001/.tez/application_1452793209987_0001/localmode-log-dir/dag_1452793209987_0001_1.dot
2016-01-14 17:40:10,132 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001/.tez/application_1452793209987_0001/localmode-log-dir/dag_1452793209987_0001_1-tez-dag.pb.txt
2016-01-14 17:40:10,146 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793209987_0001_1
2016-01-14 17:40:10,146 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:10,146 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001] test/(1/1) ...latform/testSimple/simple
2016-01-14 17:40:10 Running Dag: dag_1452793209987_0001_1
2016-01-14 17:40:10,159 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452793209987_0001_1, submitTime=1452793210130
2016-01-14 17:40:10,159 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 17:40:10,160 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A2E0E6E8DCA44AEEB68E87F86BB32C5C
2016-01-14 17:40:10,160 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 17:40:10,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452793209987_0001_1, initTime=1452793210159
2016-01-14 17:40:10,161 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 17:40:10,161 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793209987_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 17:40:10,162 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:DAG_STARTED]: dagID=dag_1452793209987_0001_1, startTime=1452793210162
2016-01-14 17:40:10,162 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 17:40:10,162 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 17:40:10,163 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793209987_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 17:40:10,163 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EEE7918D359B4CB4917B59AFED850F5C : {33BA7B0903C544B19D4E934D9051A833={InputName=33BA7B0903C544B19D4E934D9051A833}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 17:40:10,163 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 33BA7B0903C544B19D4E934D9051A833, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 17:40:10,163 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C]
2016-01-14 17:40:10,163 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C]
2016-01-14 17:40:10,163 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C]
2016-01-14 17:40:10,164 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C]. Starting root input initializers: 1
2016-01-14 17:40:10,164 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 17:40:10,164 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C]
2016-01-14 17:40:10,168 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 17:40:10,168 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C]
2016-01-14 17:40:10,169 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C]
2016-01-14 17:40:10,169 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C]
2016-01-14 17:40:10,169 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1452793209987_0001, dagName=[6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001] test/(1/1) ...latform/testSimple/simple
2016-01-14 17:40:10,172 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2847B68196B747948BAE5FF295902CB5, vertexId=vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 17:40:10,173 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2847B68196B747948BAE5FF295902CB5, vertexId=vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C]
2016-01-14 17:40:10,175 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2847B68196B747948BAE5FF295902CB5, vertexName=A2E0E6E8DCA44AEEB68E87F86BB32C5C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 17:40:10,175 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2847B68196B747948BAE5FF295902CB5, vertexId=vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C]
2016-01-14 17:40:10,177 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:10,177 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 33BA7B0903C544B19D4E934D9051A833 on vertex vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C]
2016-01-14 17:40:10,177 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A2E0E6E8DCA44AEEB68E87F86BB32C5C, vertexId=vertex_1452793209987_0001_1_01, initRequestedTime=1452793210164, initedTime=1452793210177, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 17:40:10,178 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C] transitioned from NEW to INITED due to event V_INIT
2016-01-14 17:40:10,179 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 17:40:10,182 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 33BA7B0903C544B19D4E934D9051A833 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 17:40:10,183 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 17:40:10,184 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 17:40:10,186 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 408
2016-01-14 17:40:10,187 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 33BA7B0903C544B19D4E934D9051A833 on vertex vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C]
2016-01-14 17:40:10,188 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:10,188 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {33BA7B0903C544B19D4E934D9051A833=forAllWorkUnits=true, update=[1]}
2016-01-14 17:40:10,188 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C] parallelism set to 1
2016-01-14 17:40:10,188 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 17:40:10,188 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793209987_0001_1_00, eventCount=1
2016-01-14 17:40:10,188 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EEE7918D359B4CB4917B59AFED850F5C, vertexId=vertex_1452793209987_0001_1_00, initRequestedTime=1452793210163, initedTime=1452793210188, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 17:40:10,189 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C] with distanceFromRoot: 0
2016-01-14 17:40:10,188 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 17:40:10,189 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 17:40:10,189 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EEE7918D359B4CB4917B59AFED850F5C
2016-01-14 17:40:10,189 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C]
2016-01-14 17:40:10,190 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:10,190 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793209987_0001_1_00, startRequestedTime=1452793210189, startedTime=1452793210189
2016-01-14 17:40:10,190 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:10,190 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:10,191 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:TASK_STARTED]: vertexName=EEE7918D359B4CB4917B59AFED850F5C, taskId=task_1452793209987_0001_1_00_000000, scheduledTime=1452793210190, launchTime=1452793210190
2016-01-14 17:40:10,191 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793209987_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:10,191 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452793209987_0001_1_00 for vertex: vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C] numStartedSources: 1 numSources: 1
2016-01-14 17:40:10,192 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C] with distanceFromRoot: 1
2016-01-14 17:40:10,192 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C]
2016-01-14 17:40:10,192 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EEE7918D359B4CB4917B59AFED850F5C in vertex: A2E0E6E8DCA44AEEB68E87F86BB32C5C
2016-01-14 17:40:10,192 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A2E0E6E8DCA44AEEB68E87F86BB32C5C with 1 source tasks and 1 pending tasks
2016-01-14 17:40:10,193 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 17:40:10,193 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452793209987_0001_1_01, startRequestedTime=1452793210192, startedTime=1452793210192
2016-01-14 17:40:10,193 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] submitted tez dag to app master: application_1452793209987_0001, with dag id: dag_1452793209987_0001_1
2016-01-14 17:40:10,193 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 17:40:10,194 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001] test/(1/1) ...latform/testSimple/simple, VertexName: EEE7918D359B4CB4917B59AFED850F5C, VertexParallelism: 1, TaskAttemptID:attempt_1452793209987_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=33BA7B0903C544B19D4E934D9051A833, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A2E0E6E8DCA44AEEB68E87F86BB32C5C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:10,194 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793209987_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:10,194 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:10,203 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 17:40:10,203 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 17:40:10,204 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 17:40:10,204 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793209987_0001_00_000001
2016-01-14 17:40:10,204 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:10,206 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793209987_0001_00_000001
2016-01-14 17:40:10,206 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:10,207 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:10,208 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:10,208 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793209987_0001_00_000001, launchTime=1452793210207
2016-01-14 17:40:10,209 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793209987_0001_00_000001
2016-01-14 17:40:10,210 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793209987_0001_1_00_000000_0] to container: [container_1452793209987_0001_00_000001]
2016-01-14 17:40:10,210 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:10,211 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793209987_0001_1_00_000000_0] started. Is using containerId: [container_1452793209987_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 17:40:10,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EEE7918D359B4CB4917B59AFED850F5C, taskAttemptId=attempt_1452793209987_0001_1_00_000000_0, startTime=1452793210211, containerId=container_1452793209987_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793209987_0001_00_000001/teamcity, completedLogs=
2016-01-14 17:40:10,211 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793209987_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:10,211 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793209987_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:10,212 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:10,212 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793209987_0001_00_000001 given task: attempt_1452793209987_0001_1_00_000000_0
2016-01-14 17:40:10,213 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793209987_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793209987_0001_1_00_000000_0
2016-01-14 17:40:10 Starting to run new task attempt: attempt_1452793209987_0001_1_00_000000_0
2016-01-14 17:40:10,213 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793209987_0001_1_00_000000_0
2016-01-14 17:40:10,213 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:10,213 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:10,214 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:10,214 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:10,214 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:10,214 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:10,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001] test/(1/1) ...latform/testSimple/simple, VertexName: EEE7918D359B4CB4917B59AFED850F5C, VertexParallelism: 1, TaskAttemptID:attempt_1452793209987_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=33BA7B0903C544B19D4E934D9051A833, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A2E0E6E8DCA44AEEB68E87F86BB32C5C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 17:40:10,214 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:10,216 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793209987_0001_1_00_000000_0
2016-01-14 17:40:10,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:10,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:10,217 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=33BA7B0903C544B19D4E934D9051A833, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 17:40:10,218 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:10,218 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 33BA7B0903C544B19D4E934D9051A833
2016-01-14 17:40:10,218 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 33BA7B0903C544B19D4E934D9051A833
2016-01-14 17:40:10,222 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 17:40:10,223 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 33BA7B0903C544B19D4E934D9051A833
2016-01-14 17:40:10,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 33BA7B0903C544B19D4E934D9051A833
2016-01-14 17:40:10,222 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:10,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A2E0E6E8DCA44AEEB68E87F86BB32C5C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 17:40:10,223 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793209987_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 17:40:10,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:10,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A2E0E6E8DCA44AEEB68E87F86BB32C5C
2016-01-14 17:40:10,227 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:10,228 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 17:40:10,228 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A2E0E6E8DCA44AEEB68E87F86BB32C5C
2016-01-14 17:40:10,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A2E0E6E8DCA44AEEB68E87F86BB32C5C
2016-01-14 17:40:10,233 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EEE7918D359B4CB4917B59AFED850F5C, ordinal: 0
2016-01-14 17:40:10,233 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EEE7918D359B4CB4917B59AFED850F5C, mem on start (mb), free: 412, total: 538, max: 672
2016-01-14 17:40:10,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:10,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:10,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:10,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:10,235 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:10,235 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 17:40:10,236 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:10,236 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 17:40:10,236 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:10,237 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 17:40:10,237 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 33BA7B0903C544B19D4E934D9051A833, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 17:40:10,237 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A2E0E6E8DCA44AEEB68E87F86BB32C5C, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 17:40:10,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:10,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 33BA7B0903C544B19D4E934D9051A833 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:10,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 33BA7B0903C544B19D4E934D9051A833
2016-01-14 17:40:10,238 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:10,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 33BA7B0903C544B19D4E934D9051A833
2016-01-14 17:40:10,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:10,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:10,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:10,239 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793209987_0001_1_00_000000_0
2016-01-14 17:40:10,242 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:10,242 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"] streamed: true, id: 33BA7B0903C544B19D4E934D9051A833
2016-01-14 17:40:10,242 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(test)[by:[{1}:0]], id: C8C184041F8744A59D748E8EC0109892
2016-01-14 17:40:10,243 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(test)[by:[{1}:0]] C8C184041F8744A59D748E8EC0109892
2016-01-14 17:40:10,243 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@79cb88df; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@218726f1; comparator=cascading.tuple.hadoop.util.TupleComparator@300da969; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 17:40:10,243 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 17:40:10,243 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 17:40:10,243 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:10,248 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 17:40:10,250 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7c5597c
2016-01-14 17:40:10,254 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7c5597c
2016-01-14 17:40:10,254 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 17:40:10,255 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 17:40:10,259 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 17:40:10,259 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 17:40:10,259 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 17:40:10,260 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 17:40:10,260 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 17:40:10,262 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[[0:1]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/apache.10.txt"]
2016-01-14 17:40:10,262 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 17:40:10,262 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EEE7918D359B4CB4917B59AFED850F5C, all 1 inputs ready in: 00:00:00.000
2016-01-14 17:40:10,266 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793209987_0001_1_00_000000_0
2016-01-14 17:40:10,266 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EEE7918D359B4CB4917B59AFED850F5C, mem on close (mb), free: 310, total: 538, max: 672
2016-01-14 17:40:10,266 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 17:40:10,266 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 17:40:10,266 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 210; bufvoid = 104857600
2016-01-14 17:40:10,266 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214360(104857440); length = 37/6553600
2016-01-14 17:40:10,268 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 17:40:10 Completed running task attempt: attempt_1452793209987_0001_1_00_000000_0
2016-01-14 17:40:10,272 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793209987_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:10,273 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452793209987_0001_1_00, eventCount=1
2016-01-14 17:40:10,273 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:10,273 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EEE7918D359B4CB4917B59AFED850F5C, taskAttemptId=attempt_1452793209987_0001_1_00_000000_0, startTime=1452793210211, finishTime=1452793210273, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=564133888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793210262, Process_Duration=3, Process_End_Time=1452793210265, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 17:40:10,273 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793209987_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:10,274 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793209987_0001_1_00_000000_0
2016-01-14 17:40:10,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:10,274 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:TASK_FINISHED]: vertexName=EEE7918D359B4CB4917B59AFED850F5C, taskId=task_1452793209987_0001_1_00_000000, startTime=1452793210211, finishTime=1452793210274, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1452793209987_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=564133888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793210262, Process_Duration=3, Process_End_Time=1452793210265, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10
2016-01-14 17:40:10,274 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793209987_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:10,275 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C] : 1
2016-01-14 17:40:10,275 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C]
2016-01-14 17:40:10,276 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:10,276 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:10,277 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:VERTEX_FINISHED]: vertexName=EEE7918D359B4CB4917B59AFED850F5C, vertexId=vertex_1452793209987_0001_1_00, initRequestedTime=1452793210163, initedTime=1452793210188, startRequestedTime=1452793210189, startedTime=1452793210189, finishTime=1452793210275, timeTaken=86, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=564133888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793210262, Process_Duration=3, Process_End_Time=1452793210265, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, vertexStats=firstTaskStartTime=1452793210211, firstTasksToStart=[ task_1452793209987_0001_1_00_000000 ], lastTaskFinishTime=1452793210273, lastTasksToFinish=[ task_1452793209987_0001_1_00_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793209987_0001_1_00_000000 ], longestDurationTasks=[ task_1452793209987_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:10,278 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:10,278 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C] attempt: attempt_1452793209987_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 17:40:10,278 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A2E0E6E8DCA44AEEB68E87F86BB32C5C
2016-01-14 17:40:10,278 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C]
2016-01-14 17:40:10,279 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:10,279 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:10,279 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:TASK_STARTED]: vertexName=A2E0E6E8DCA44AEEB68E87F86BB32C5C, taskId=task_1452793209987_0001_1_01_000000, scheduledTime=1452793210279, launchTime=1452793210279
2016-01-14 17:40:10,279 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793209987_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 17:40:10,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793209987_0001_00_000001, stoppedTime=1452793210280, exitStatus=0
2016-01-14 17:40:10,279 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 17:40:10,280 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:10,281 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:10,281 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C]
2016-01-14 17:40:10,281 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452793209987_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 17:40:10,281 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001] test/(1/1) ...latform/testSimple/simple, VertexName: A2E0E6E8DCA44AEEB68E87F86BB32C5C, VertexParallelism: 1, TaskAttemptID:attempt_1452793209987_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EEE7918D359B4CB4917B59AFED850F5C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2847B68196B747948BAE5FF295902CB5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:10,282 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793209987_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 17:40:10,282 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 17:40:10,282 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 17:40:10,283 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452793209987_0001_00_000002
2016-01-14 17:40:10,283 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 17:40:10,285 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452793209987_0001_00_000002
2016-01-14 17:40:10,285 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 17:40:10,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=2206, FILE_BYTES_WRITTEN=236, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=10, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=564133888, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=10, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452793210262, Process_Duration=3, Process_End_Time=1452793210265, Read_Duration=2, Tuples_Read=10, Tuples_Written=10, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=10]]
2016-01-14 17:40:10,286 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:10,286 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:10,286 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793209987_0001_00_000001
2016-01-14 17:40:10,287 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793209987_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 17:40:10,287 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793209987_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:10,287 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793209987_0001_00_000001, Exiting
2016-01-14 17:40:10,287 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793209987_0001_00_000001
2016-01-14 17:40:10,287 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793209987_0001_00_000001
2016-01-14 17:40:10,288 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 17:40:10,288 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 17:40:10,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452793209987_0001_00_000002, launchTime=1452793210288
2016-01-14 17:40:10,289 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793209987_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:10,290 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793209987_0001_00_000001 completed successfully
2016-01-14 17:40:10,290 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 17:40:10,290 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793209987_0001_00_000002
2016-01-14 17:40:10,291 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452793209987_0001_1_01_000000_0] to container: [container_1452793209987_0001_00_000002]
2016-01-14 17:40:10,291 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 17:40:10,292 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452793209987_0001_1_01_000000_0] started. Is using containerId: [container_1452793209987_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 17:40:10,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A2E0E6E8DCA44AEEB68E87F86BB32C5C, taskAttemptId=attempt_1452793209987_0001_1_01_000000_0, startTime=1452793210292, containerId=container_1452793209987_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452793209987_0001_00_000002/teamcity, completedLogs=
2016-01-14 17:40:10,292 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793209987_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 17:40:10,292 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793209987_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 17:40:10,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 17:40:10,293 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452793209987_0001_00_000002 given task: attempt_1452793209987_0001_1_01_000000_0
2016-01-14 17:40:10,293 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793209987_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452793209987_0001_1_01_000000_0
2016-01-14 17:40:10 Starting to run new task attempt: attempt_1452793209987_0001_1_01_000000_0
2016-01-14 17:40:10,294 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452793209987_0001_1_01_000000_0
2016-01-14 17:40:10,294 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:10,294 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 17:40:10,294 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 17:40:10,294 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 17:40:10,294 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 17:40:10,295 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 17:40:10,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001] test/(1/1) ...latform/testSimple/simple, VertexName: A2E0E6E8DCA44AEEB68E87F86BB32C5C, VertexParallelism: 1, TaskAttemptID:attempt_1452793209987_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EEE7918D359B4CB4917B59AFED850F5C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2847B68196B747948BAE5FF295902CB5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 17:40:10,295 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 17:40:10,296 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452793209987_0001_1_01_000000_0
2016-01-14 17:40:10,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 17:40:10,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:10,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EEE7918D359B4CB4917B59AFED850F5C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 17:40:10,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 17:40:10,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EEE7918D359B4CB4917B59AFED850F5C
2016-01-14 17:40:10,300 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 17:40:10,300 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 17:40:10,301 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EEE7918D359B4CB4917B59AFED850F5C
2016-01-14 17:40:10,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EEE7918D359B4CB4917B59AFED850F5C
2016-01-14 17:40:10,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:10,305 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 17:40:10,311 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452793209987_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 17:40:10,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2847B68196B747948BAE5FF295902CB5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 17:40:10,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 17:40:10,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2847B68196B747948BAE5FF295902CB5
2016-01-14 17:40:10,312 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 17:40:10,312 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2847B68196B747948BAE5FF295902CB5
2016-01-14 17:40:10,322 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A2E0E6E8DCA44AEEB68E87F86BB32C5C, ordinal: 1
2016-01-14 17:40:10,322 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2E0E6E8DCA44AEEB68E87F86BB32C5C, mem on start (mb), free: 305, total: 538, max: 672
2016-01-14 17:40:10,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 17:40:10,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 17:40:10,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 17:40:10,330 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 17:40:10,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2847B68196B747948BAE5FF295902CB5
2016-01-14 17:40:10,331 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 17:40:10,331 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 17:40:10,331 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 17:40:10,331 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 17:40:10,331 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 17:40:10,331 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 17:40:10,332 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 17:40:10,332 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EEE7918D359B4CB4917B59AFED850F5C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 17:40:10,332 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2847B68196B747948BAE5FF295902CB5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 17:40:10,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 17:40:10,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EEE7918D359B4CB4917B59AFED850F5C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 17:40:10,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 17:40:10,332 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 17:40:10,333 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EEE7918D359B4CB4917B59AFED850F5C
2016-01-14 17:40:10,333 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 17:40:10,334 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 17:40:10,334 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EEE7918D359B4CB4917B59AFED850F5C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 17:40:10,334 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 17:40:10,334 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 17:40:10,335 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 17:40:10,337 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EEE7918D359B4CB4917B59AFED850F5C
2016-01-14 17:40:10,339 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 17:40:10,339 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452793209987_0001_1_01_000000_0
2016-01-14 17:40:10,339 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452793209987_0001_1_00_000000_0_10089, runDuration: 43649, ]
2016-01-14 17:40:10,342 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(test)[by:[{1}:0]]
2016-01-14 17:40:10,343 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(test)[by:[{1}:0]] streamed: true, id: C8C184041F8744A59D748E8EC0109892
2016-01-14 17:40:10,343 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"], id: 2847B68196B747948BAE5FF295902CB5
2016-01-14 17:40:10,343 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[[0]->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/basicpipesplatform/testSimple/simple"]
2016-01-14 17:40:10,344 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(test)[by:[{1}:0]] C8C184041F8744A59D748E8EC0109892, for 1 inputs
2016-01-14 17:40:10,344 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452793209987_0001_1_00_000000_0_10089] to DISK_DIRECT, CompressedSize=204, DecompressedSize=200,EndTime=1452793210344, TimeTaken=1, Rate=0.19 MB/s
2016-01-14 17:40:10,345 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EEE7918D359B4CB4917B59AFED850F5C
2016-01-14 17:40:10,345 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 17:40:10,345 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [EEE7918D359B4CB4917B59AFED850F5C] #14 in 2ms
2016-01-14 17:40:10,345 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EEE7918D359B4CB4917B59AFED850F5C] #14, status:false, isInterrupted:false
2016-01-14 17:40:10,346 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 17:40:10,347 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 204 bytes from disk
2016-01-14 17:40:10,347 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:10,347 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 17:40:10,347 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 175 bytes
2016-01-14 17:40:10,347 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EEE7918D359B4CB4917B59AFED850F5C
2016-01-14 17:40:10,347 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 17:40:10,348 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A2E0E6E8DCA44AEEB68E87F86BB32C5C, all 1 inputs ready in: 00:00:00.004
2016-01-14 17:40:10,350 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@54e60dbb; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4b8f9b88
2016-01-14 17:40:10,352 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 17:40:10,352 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 17:40:10,353 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452793209987_0001_1_01_000000_0
2016-01-14 17:40:10,353 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452793209987_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 17:40:10,354 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452793209987_0001_1_01_000000_0
2016-01-14 17:40:10,354 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2E0E6E8DCA44AEEB68E87F86BB32C5C, mem on close (mb), free: 302, total: 538, max: 672
2016-01-14 17:40:10,355 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EEE7918D359B4CB4917B59AFED850F5C
2016-01-14 17:40:10,355 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 17:40:10,356 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 17:40:10,356 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 17:40:10,357 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 17:40:10,357 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 17:40:10 Completed running task attempt: attempt_1452793209987_0001_1_01_000000_0
2016-01-14 17:40:10,359 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452793209987_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 17:40:10,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A2E0E6E8DCA44AEEB68E87F86BB32C5C, taskAttemptId=attempt_1452793209987_0001_1_01_000000_0, startTime=1452793210292, finishTime=1452793210360, timeTaken=68, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=564133888, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793210348, Process_Duration=6, Process_End_Time=1452793210354, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 17:40:10,361 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452793209987_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 17:40:10,361 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452793209987_0001_1_01_000000_0
2016-01-14 17:40:10,361 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 17:40:10,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:TASK_FINISHED]: vertexName=A2E0E6E8DCA44AEEB68E87F86BB32C5C, taskId=task_1452793209987_0001_1_01_000000, startTime=1452793210292, finishTime=1452793210361, timeTaken=69, status=SUCCEEDED, successfulAttemptID=attempt_1452793209987_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=564133888, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793210348, Process_Duration=6, Process_End_Time=1452793210354, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8
2016-01-14 17:40:10,362 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452793209987_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 17:40:10,362 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C] : 1
2016-01-14 17:40:10,362 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C]
2016-01-14 17:40:10,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:VERTEX_FINISHED]: vertexName=A2E0E6E8DCA44AEEB68E87F86BB32C5C, vertexId=vertex_1452793209987_0001_1_01, initRequestedTime=1452793210164, initedTime=1452793210177, startRequestedTime=1452793210192, startedTime=1452793210192, finishTime=1452793210362, timeTaken=170, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=564133888, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452793210348, Process_Duration=6, Process_End_Time=1452793210354, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=8, vertexStats=firstTaskStartTime=1452793210292, firstTasksToStart=[ task_1452793209987_0001_1_01_000000 ], lastTaskFinishTime=1452793210360, lastTasksToFinish=[ task_1452793209987_0001_1_01_000000 ], minTaskDuration=68, maxTaskDuration=68, avgTaskDuration=68.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452793209987_0001_1_01_000000 ], longestDurationTasks=[ task_1452793209987_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 17:40:10,363 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 17:40:10,364 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 17:40:10,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 17:40:10,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 17:40:10,366 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 17:40:10,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452793209987_0001_00_000002, stoppedTime=1452793210367, exitStatus=0
2016-01-14 17:40:10,367 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 17:40:10,367 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 17:40:10,367 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452793209987_0001_1
2016-01-14 17:40:10,367 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 17:40:10,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452793209987_0001_1
2016-01-14 17:40:10,368 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452793209987_0001_1_00 [EEE7918D359B4CB4917B59AFED850F5C]
2016-01-14 17:40:10,368 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2847B68196B747948BAE5FF295902CB5 for vertex: vertex_1452793209987_0001_1_01 [A2E0E6E8DCA44AEEB68E87F86BB32C5C]
2016-01-14 17:40:10,372 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=236, FILE_BYTES_WRITTEN=140, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=10, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=564133888, OUTPUT_RECORDS=8, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452793210348, Process_Duration=6, Process_End_Time=1452793210354, Read_Duration=0, Tuples_Read=10, Tuples_Written=8, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=8]]
2016-01-14 17:40:10,372 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 17:40:10,372 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:10,372 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452793209987_0001_00_000002
2016-01-14 17:40:10,373 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452793209987_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 17:40:10,373 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452793209987_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 17:40:10,373 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452793209987_0001_00_000002, Exiting
2016-01-14 17:40:10,373 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452793209987_0001_00_000002
2016-01-14 17:40:10,373 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452793209987_0001_00_000002
2016-01-14 17:40:10,374 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452793209987_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 17:40:10,374 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452793209987_0001_00_000002 completed successfully
2016-01-14 17:40:10,381 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452793209987_0001_1, queueSize=0
2016-01-14 17:40:10,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452793209987_0001_1][Event:DAG_FINISHED]: dagId=dag_1452793209987_0001_1, startTime=1452793210162, finishTime=1452793210379, timeTaken=217, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=2442, FILE_BYTES_WRITTEN=376, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=7, REDUCE_INPUT_RECORDS=10, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1128267776, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=18, OUTPUT_BYTES=210, OUTPUT_BYTES_WITH_OVERHEAD=200, OUTPUT_BYTES_PHYSICAL=204, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=204, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=204, SHUFFLE_BYTES_DECOMPRESSED=200, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=204, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=11, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905586420610, Process_Duration=9, Process_End_Time=2905586420619, Read_Duration=2, Tuples_Read=20, Tuples_Written=18, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=10, Tuples_Written=8
2016-01-14 17:40:10,382 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452793209987_0001_1 finished with state: SUCCEEDED
2016-01-14 17:40:10,382 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 17:40:10,383 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452793209987_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 17:40:10,383 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452793209987_0001_00_000002
2016-01-14 17:40:10,383 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452793209987_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 17:40:10 Completed Dag: dag_1452793209987_0001_1
2016-01-14 17:40:10,384 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452793209987_0001_1, dagState=SUCCEEDED
2016-01-14 17:40:10,384 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452793209987_0001_1_post
2016-01-14 17:40:10,384 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 17:40:10,385 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 17:40:10,391 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/testSimple/simple, applicationId=application_1452793209987_0001
2016-01-14 17:40:10,393 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 17:40:10,393 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 17:40:10,393 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 17:40:10,393 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 17:40:10,393 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 17:40:10,394 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 17:40:10,394 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 17:40:10,394 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 17:40:10,394 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 17:40:10,395 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 17:40:10,395 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 17:40:10,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 17:40:10,395 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 17:40:10,400 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6E83D560018B41358434EBD72436B8A9/A64966A5E00E4105897FA9EB21F63001/.tez/application_1452793209987_0001
2016-01-14 17:40:10,400 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [test] completed in: 00:00.434
2016-01-14 17:40:10,462 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/basicpipesplatform/testSimple/simple