2016-01-14 18:31:09,903 INFO platform.PlatformRunner (PlatformRunner.java:getPlatformClassesFromClasspath(195)) - found 1 test platforms from classpath
2016-01-14 18:31:09,928 INFO platform.PlatformRunner (PlatformRunner.java:addPlatform(244)) - adding test: cascading.detail.EveryPipeAssemblyPlatformTest, with platform: hadoop2-tez
2016-01-14 18:31:10,390 INFO tez.Hadoop2TezPlatform (Hadoop2TezPlatform.java:setUp(136)) - not using cluster
2016-01-14 18:31:10,821 WARN util.NativeCodeLoader (NativeCodeLoader.java:<clinit>(62)) - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-01-14 18:31:11,089 INFO platform.TestPlatform (TestPlatform.java:getGlobalProperties(73)) - platform property overrides:
2016-01-14 18:31:11,254 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:11,256 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:11,346 INFO property.AppProps (AppProps.java:getAppID(177)) - using app.id: BB25FF09AF2143D8B57A035B8F9607FE
2016-01-14 18:31:11,616 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.078
2016-01-14 18:31:11,637 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.100
2016-01-14 18:31:11,638 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:11,639 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:11,640 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:11,785 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:11,916 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1
2016-01-14 18:31:12,147 INFO util.Version (Version.java:printBanner(82)) - Concurrent, Inc - Cascading 3.0.4
2016-01-14 18:31:12,150 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:12,150 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:12,151 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1"]
2016-01-14 18:31:12,151 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:12,151 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:12,151 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:12,152 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/-1_'field'_-1
2016-01-14 18:31:12,153 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:12,180 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:12,343 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:12,350 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:12,351 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:12,404 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D/.tez/application_1452796272350_0001 doesn't exist and is created
2016-01-14 18:31:12,481 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:12,481 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:12,482 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D/.tez/application_1452796272350_0001
2016-01-14 18:31:12,538 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796272350_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:12,579 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:12,582 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: NEW
2016-01-14 18:31:12,582 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(238)) - DAGAppMaster is not started wait for 100ms...
2016-01-14 18:31:12,583 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:12,594 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:12,617 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:12,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:12,623 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D/.tez/application_1452796272350_0001/localmode-log-dir
2016-01-14 18:31:12,623 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D/.tez/application_1452796272350_0001/localmode-log-dir/history.txt.appattempt_1452796272350_0001_000000, maxErrors=10
2016-01-14 18:31:12,624 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:12,625 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796272350_0001_000000, appSubmitTime=1452796272522, launchTime=1452796272537
2016-01-14 18:31:12,632 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:12,632 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:12,634 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:12,634 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:12,683 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 18:31:12,683 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 18:31:12,683 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:12,696 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:46198
2016-01-14 18:31:12,724 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:12,784 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: INITED
2016-01-14 18:31:12,784 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(240)) - DAGAppMaster is not startetd wait for 100ms...
2016-01-14 18:31:12,784 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:12,806 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:12,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796272350_0001_000000, startTime=1452796272805
2016-01-14 18:31:12,806 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:12,884 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:12,901 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:12,901 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1452796272350_0001, dagName=[7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D] every/(1/1) ...blyplatform/-1_'field'_-1
2016-01-14 18:31:13,157 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D] every/(1/1) ...blyplatform/-1_'field'_-1
2016-01-14 18:31:13,256 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796272350_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D/.tez/application_1452796272350_0001/localmode-log-dir/dag_1452796272350_0001_1.dot
2016-01-14 18:31:13,260 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D/.tez/application_1452796272350_0001/localmode-log-dir/dag_1452796272350_0001_1-tez-dag.pb.txt
2016-01-14 18:31:13,373 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796272350_0001_1
2016-01-14 18:31:13,373 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:13,373 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D] every/(1/1) ...blyplatform/-1_'field'_-1
2016-01-14 18:31:13 Running Dag: dag_1452796272350_0001_1
2016-01-14 18:31:13,401 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:13,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796272350_0001_1, submitTime=1452796273157
2016-01-14 18:31:13,432 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E0B95840A3B345E38C5093233C7FFD4B
2016-01-14 18:31:13,440 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:13,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:13,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796272350_0001_1, initTime=1452796273403
2016-01-14 18:31:13,444 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796272350_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:13,449 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:13,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:DAG_STARTED]: dagID=dag_1452796272350_0001_1, startTime=1452796273447
2016-01-14 18:31:13,449 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:13,452 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796272350_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:13,453 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D7DF6DB1B8CC4837B300A031CC2F8E7C : {A42CC1DCFD944851879B578E52BF0446={InputName=A42CC1DCFD944851879B578E52BF0446}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:13,455 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: A42CC1DCFD944851879B578E52BF0446, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:13,455 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C]
2016-01-14 18:31:13,462 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C]
2016-01-14 18:31:13,462 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C]
2016-01-14 18:31:13,462 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1452796272350_0001, dagName=[7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D] every/(1/1) ...blyplatform/-1_'field'_-1
2016-01-14 18:31:13,464 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C]. Starting root input initializers: 1
2016-01-14 18:31:13,479 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: A42CC1DCFD944851879B578E52BF0446 on vertex vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C]
2016-01-14 18:31:13,484 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:13,495 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B]
2016-01-14 18:31:13,509 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796272350_0001, with dag id: dag_1452796272350_0001_1
2016-01-14 18:31:13,516 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:13,518 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input A42CC1DCFD944851879B578E52BF0446 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:13,519 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:13,520 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:13,534 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:13,534 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B]
2016-01-14 18:31:13,552 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B]
2016-01-14 18:31:13,554 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B]
2016-01-14 18:31:13,554 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=79BB2AACFB004469AB2DA45DE06D94BE, vertexId=vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:13,558 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=79BB2AACFB004469AB2DA45DE06D94BE, vertexId=vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B]
2016-01-14 18:31:13,567 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:13,569 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: A42CC1DCFD944851879B578E52BF0446 on vertex vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C]
2016-01-14 18:31:13,577 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=79BB2AACFB004469AB2DA45DE06D94BE, vertexName=E0B95840A3B345E38C5093233C7FFD4B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:13,588 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=79BB2AACFB004469AB2DA45DE06D94BE, vertexId=vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B]
2016-01-14 18:31:13,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E0B95840A3B345E38C5093233C7FFD4B, vertexId=vertex_1452796272350_0001_1_01, initRequestedTime=1452796273484, initedTime=1452796273590, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:13,592 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:13,597 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:13,601 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:13,601 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {A42CC1DCFD944851879B578E52BF0446=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:13,604 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C] parallelism set to 1
2016-01-14 18:31:13,604 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:13,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796272350_0001_1_00, eventCount=1
2016-01-14 18:31:13,615 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:13,616 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D7DF6DB1B8CC4837B300A031CC2F8E7C, vertexId=vertex_1452796272350_0001_1_00, initRequestedTime=1452796273453, initedTime=1452796273615, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:13,617 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C] with distanceFromRoot: 0
2016-01-14 18:31:13,617 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:13,618 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D7DF6DB1B8CC4837B300A031CC2F8E7C
2016-01-14 18:31:13,618 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C]
2016-01-14 18:31:13,620 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:13,621 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796272350_0001_1_00, startRequestedTime=1452796273617, startedTime=1452796273617
2016-01-14 18:31:13,622 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:13,651 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:13,652 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:TASK_STARTED]: vertexName=D7DF6DB1B8CC4837B300A031CC2F8E7C, taskId=task_1452796272350_0001_1_00_000000, scheduledTime=1452796273650, launchTime=1452796273650
2016-01-14 18:31:13,652 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796272350_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:13,653 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796272350_0001_1_00 for vertex: vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B] numStartedSources: 1 numSources: 1
2016-01-14 18:31:13,653 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B] with distanceFromRoot: 1
2016-01-14 18:31:13,657 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C]
2016-01-14 18:31:13,661 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D7DF6DB1B8CC4837B300A031CC2F8E7C in vertex: E0B95840A3B345E38C5093233C7FFD4B
2016-01-14 18:31:13,663 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E0B95840A3B345E38C5093233C7FFD4B with 1 source tasks and 1 pending tasks
2016-01-14 18:31:13,664 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:13,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796272350_0001_1_01, startRequestedTime=1452796273658, startedTime=1452796273658
2016-01-14 18:31:13,664 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:13,666 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: D7DF6DB1B8CC4837B300A031CC2F8E7C, VertexParallelism: 1, TaskAttemptID:attempt_1452796272350_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A42CC1DCFD944851879B578E52BF0446, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E0B95840A3B345E38C5093233C7FFD4B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:13,669 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796272350_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:13,669 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:13,711 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:13,714 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:13,714 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:13,721 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796272350_0001_00_000001
2016-01-14 18:31:13,721 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:13,729 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796272350_0001_00_000001
2016-01-14 18:31:13,729 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:13,735 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:13,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796272350_0001_00_000001, launchTime=1452796273734
2016-01-14 18:31:13,736 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:13,738 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796272350_0001_00_000001
2016-01-14 18:31:13,741 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796272350_0001_1_00_000000_0] to container: [container_1452796272350_0001_00_000001]
2016-01-14 18:31:13,742 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:13,743 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796272350_0001_00_000001 given task: attempt_1452796272350_0001_1_00_000000_0
2016-01-14 18:31:13,743 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796272350_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796272350_0001_1_00_000000_0
2016-01-14 18:31:13,744 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796272350_0001_1_00_000000_0] started. Is using containerId: [container_1452796272350_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:13 Starting to run new task attempt: attempt_1452796272350_0001_1_00_000000_0
2016-01-14 18:31:13,745 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796272350_0001_1_00_000000_0
2016-01-14 18:31:13,745 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:13,746 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:13,746 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D7DF6DB1B8CC4837B300A031CC2F8E7C, taskAttemptId=attempt_1452796272350_0001_1_00_000000_0, startTime=1452796273743, containerId=container_1452796272350_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796272350_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:13,747 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796272350_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:13,748 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:13,748 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:13,749 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:13,751 INFO counters.Limits (Limits.java:ensureInitialized(59)) - Counter limits initialized with parameters: GROUP_NAME_MAX=128, MAX_GROUPS=500, COUNTER_NAME_MAX=64, MAX_COUNTERS=1200
2016-01-14 18:31:13,751 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:13,755 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796272350_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:13,774 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:13,775 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: D7DF6DB1B8CC4837B300A031CC2F8E7C, VertexParallelism: 1, TaskAttemptID:attempt_1452796272350_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A42CC1DCFD944851879B578E52BF0446, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E0B95840A3B345E38C5093233C7FFD4B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:13,778 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:13,785 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796272350_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:13,787 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796272350_0001_1_00_000000_0
2016-01-14 18:31:13,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:13,788 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:13,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A42CC1DCFD944851879B578E52BF0446, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:13,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:13,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:13,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A42CC1DCFD944851879B578E52BF0446
2016-01-14 18:31:13,799 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: A42CC1DCFD944851879B578E52BF0446
2016-01-14 18:31:13,800 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E0B95840A3B345E38C5093233C7FFD4B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:13,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:13,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E0B95840A3B345E38C5093233C7FFD4B
2016-01-14 18:31:13,814 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:13,821 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: A42CC1DCFD944851879B578E52BF0446
2016-01-14 18:31:13,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A42CC1DCFD944851879B578E52BF0446
2016-01-14 18:31:13,824 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:13,825 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:13,827 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E0B95840A3B345E38C5093233C7FFD4B
2016-01-14 18:31:13,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E0B95840A3B345E38C5093233C7FFD4B
2016-01-14 18:31:13,837 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D7DF6DB1B8CC4837B300A031CC2F8E7C, ordinal: 0
2016-01-14 18:31:13,838 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D7DF6DB1B8CC4837B300A031CC2F8E7C, mem on start (mb), free: 212, total: 281, max: 672
2016-01-14 18:31:13,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:13,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:13,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:13,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:13,842 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:13,846 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:13,847 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:13,847 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:13,847 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:13,848 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:13,848 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A42CC1DCFD944851879B578E52BF0446, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:13,848 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E0B95840A3B345E38C5093233C7FFD4B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:13,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:13,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A42CC1DCFD944851879B578E52BF0446
2016-01-14 18:31:13,850 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:13,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A42CC1DCFD944851879B578E52BF0446
2016-01-14 18:31:13,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A42CC1DCFD944851879B578E52BF0446 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:13,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:13,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:13,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:13,853 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796272350_0001_1_00_000000_0
2016-01-14 18:31:13,855 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:13,865 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@22016455
2016-01-14 18:31:13,877 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@22016455
2016-01-14 18:31:13,877 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:13,877 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:13,891 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:13,907 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: A42CC1DCFD944851879B578E52BF0446
2016-01-14 18:31:13,908 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: B12C04EF45AB456EA82EB52983C09E05
2016-01-14 18:31:13,909 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] B12C04EF45AB456EA82EB52983C09E05
2016-01-14 18:31:13,933 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2b04f602; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@52b3135e; comparator=cascading.tuple.hadoop.util.TupleComparator@6abcaabc; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:13,938 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:13,941 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:13,942 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:14,081 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:14,082 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:14,082 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:14,082 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:14,083 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:14,088 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:14,089 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:14,091 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D7DF6DB1B8CC4837B300A031CC2F8E7C, all 1 inputs ready in: 00:00:00.002
2016-01-14 18:31:14,111 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796272350_0001_1_00_000000_0
2016-01-14 18:31:14,112 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D7DF6DB1B8CC4837B300A031CC2F8E7C, mem on close (mb), free: 100, total: 281, max: 672
2016-01-14 18:31:14,112 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:14,112 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:14,113 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:14,113 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:14,124 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:14 Completed running task attempt: attempt_1452796272350_0001_1_00_000000_0
2016-01-14 18:31:14,143 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796272350_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:14,144 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796272350_0001_1_00, eventCount=1
2016-01-14 18:31:14,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796274091, Process_Duration=20, Process_End_Time=1452796274111, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:14,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:14,150 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:14,158 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:14,158 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796272350_0001_00_000001
2016-01-14 18:31:14,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D7DF6DB1B8CC4837B300A031CC2F8E7C, taskAttemptId=attempt_1452796272350_0001_1_00_000000_0, startTime=1452796273743, finishTime=1452796274144, timeTaken=401, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796274091, Process_Duration=20, Process_End_Time=1452796274111, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:14,164 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796272350_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:14,164 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:14,166 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796272350_0001_1_00_000000_0
2016-01-14 18:31:14,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:14,176 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:TASK_FINISHED]: vertexName=D7DF6DB1B8CC4837B300A031CC2F8E7C, taskId=task_1452796272350_0001_1_00_000000, startTime=1452796273743, finishTime=1452796274168, timeTaken=425, status=SUCCEEDED, successfulAttemptID=attempt_1452796272350_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796274091, Process_Duration=20, Process_End_Time=1452796274111, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:14,178 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796272350_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:14,181 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:14,182 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C] : 1
2016-01-14 18:31:14,182 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C]
2016-01-14 18:31:14,192 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:14,196 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:VERTEX_FINISHED]: vertexName=D7DF6DB1B8CC4837B300A031CC2F8E7C, vertexId=vertex_1452796272350_0001_1_00, initRequestedTime=1452796273453, initedTime=1452796273615, startRequestedTime=1452796273617, startedTime=1452796273617, finishTime=1452796274182, timeTaken=565, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796274091, Process_Duration=20, Process_End_Time=1452796274111, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796273743, firstTasksToStart=[ task_1452796272350_0001_1_00_000000 ], lastTaskFinishTime=1452796274144, lastTasksToFinish=[ task_1452796272350_0001_1_00_000000 ], minTaskDuration=401, maxTaskDuration=401, avgTaskDuration=401.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796272350_0001_1_00_000000 ], longestDurationTasks=[ task_1452796272350_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:14,198 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:14,201 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:14,201 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796272350_0001_00_000001, stoppedTime=1452796274201, exitStatus=0
2016-01-14 18:31:14,203 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:14,204 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B] attempt: attempt_1452796272350_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:14,205 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E0B95840A3B345E38C5093233C7FFD4B
2016-01-14 18:31:14,205 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B]
2016-01-14 18:31:14,208 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:14,209 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796272350_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:14,210 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:14,210 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:TASK_STARTED]: vertexName=E0B95840A3B345E38C5093233C7FFD4B, taskId=task_1452796272350_0001_1_01_000000, scheduledTime=1452796274209, launchTime=1452796274209
2016-01-14 18:31:14,211 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796272350_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:14,212 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:14,212 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B]
2016-01-14 18:31:14,214 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: E0B95840A3B345E38C5093233C7FFD4B, VertexParallelism: 1, TaskAttemptID:attempt_1452796272350_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7DF6DB1B8CC4837B300A031CC2F8E7C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=79BB2AACFB004469AB2DA45DE06D94BE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:14,214 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796272350_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:14,216 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:14,218 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796272350_0001_00_000002
2016-01-14 18:31:14,218 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:14,223 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796272350_0001_00_000002
2016-01-14 18:31:14,223 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:14,224 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:14,224 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796272350_0001_00_000002, launchTime=1452796274224
2016-01-14 18:31:14,225 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:14,225 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796272350_0001_00_000002
2016-01-14 18:31:14,225 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796272350_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:14,226 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796272350_0001_00_000001: 65 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:14,226 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796272350_0001_00_000001, Exiting
2016-01-14 18:31:14,226 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796272350_0001_00_000001
2016-01-14 18:31:14,226 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796272350_0001_00_000001
2016-01-14 18:31:14,231 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796272350_0001_1_01_000000_0] to container: [container_1452796272350_0001_00_000002]
2016-01-14 18:31:14,231 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:14,232 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796272350_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:14,233 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796272350_0001_00_000001 completed successfully
2016-01-14 18:31:14,233 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796272350_0001_1_01_000000_0] started. Is using containerId: [container_1452796272350_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:14,235 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:14,235 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796272350_0001_00_000002 given task: attempt_1452796272350_0001_1_01_000000_0
2016-01-14 18:31:14,235 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796272350_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796272350_0001_1_01_000000_0
2016-01-14 18:31:14,235 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E0B95840A3B345E38C5093233C7FFD4B, taskAttemptId=attempt_1452796272350_0001_1_01_000000_0, startTime=1452796274232, containerId=container_1452796272350_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796272350_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:14,235 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796272350_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:14 Starting to run new task attempt: attempt_1452796272350_0001_1_01_000000_0
2016-01-14 18:31:14,236 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:14,236 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796272350_0001_1_01_000000_0
2016-01-14 18:31:14,236 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:14,237 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:14,237 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:14,237 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:14,237 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:14,236 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796272350_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:14,237 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:14,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D] every/(1/1) ...blyplatform/-1_'field'_-1, VertexName: E0B95840A3B345E38C5093233C7FFD4B, VertexParallelism: 1, TaskAttemptID:attempt_1452796272350_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D7DF6DB1B8CC4837B300A031CC2F8E7C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=79BB2AACFB004469AB2DA45DE06D94BE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:14,238 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:14,240 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796272350_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:14,241 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796272350_0001_1_01_000000_0
2016-01-14 18:31:14,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:14,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:14,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D7DF6DB1B8CC4837B300A031CC2F8E7C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:14,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:14,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=79BB2AACFB004469AB2DA45DE06D94BE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:14,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:14,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 79BB2AACFB004469AB2DA45DE06D94BE
2016-01-14 18:31:14,248 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:14,249 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 79BB2AACFB004469AB2DA45DE06D94BE
2016-01-14 18:31:14,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:14,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D7DF6DB1B8CC4837B300A031CC2F8E7C
2016-01-14 18:31:14,264 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:14,280 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:14,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 79BB2AACFB004469AB2DA45DE06D94BE
2016-01-14 18:31:14,285 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E0B95840A3B345E38C5093233C7FFD4B, ordinal: 1
2016-01-14 18:31:14,286 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0B95840A3B345E38C5093233C7FFD4B, mem on start (mb), free: 87, total: 281, max: 672
2016-01-14 18:31:14,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:14,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:14,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:14,288 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:14,288 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:14,288 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D7DF6DB1B8CC4837B300A031CC2F8E7C
2016-01-14 18:31:14,289 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D7DF6DB1B8CC4837B300A031CC2F8E7C
2016-01-14 18:31:14,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:14,290 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:14,290 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:14,291 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:14,291 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:14,291 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:14,291 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:14,291 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 79BB2AACFB004469AB2DA45DE06D94BE, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:14,291 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D7DF6DB1B8CC4837B300A031CC2F8E7C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:14,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:14,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D7DF6DB1B8CC4837B300A031CC2F8E7C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:14,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D7DF6DB1B8CC4837B300A031CC2F8E7C
2016-01-14 18:31:14,292 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:14,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:14,304 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:14,305 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:14,309 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D7DF6DB1B8CC4837B300A031CC2F8E7C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:14,313 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:14,313 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:14,317 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:14,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D7DF6DB1B8CC4837B300A031CC2F8E7C
2016-01-14 18:31:14,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:14,326 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796272350_0001_1_01_000000_0
2016-01-14 18:31:14,328 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796272350_0001_1_00_000000_0_10003, runDuration: 316302, ]
2016-01-14 18:31:14,333 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:14,343 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796272350_0001_1_00_000000_0_10003] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796274343, TimeTaken=6, Rate=0.05 MB/s
2016-01-14 18:31:14,343 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D7DF6DB1B8CC4837B300A031CC2F8E7C
2016-01-14 18:31:14,344 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:14,353 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D7DF6DB1B8CC4837B300A031CC2F8E7C] #1 in 17ms
2016-01-14 18:31:14,353 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D7DF6DB1B8CC4837B300A031CC2F8E7C] #1, status:false, isInterrupted:false
2016-01-14 18:31:14,355 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:14,356 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: B12C04EF45AB456EA82EB52983C09E05
2016-01-14 18:31:14,356 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1"], id: 79BB2AACFB004469AB2DA45DE06D94BE
2016-01-14 18:31:14,357 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_-1"]
2016-01-14 18:31:14,359 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] B12C04EF45AB456EA82EB52983C09E05, for 1 inputs
2016-01-14 18:31:14,360 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:14,361 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:14,367 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:14,371 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:14,371 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D7DF6DB1B8CC4837B300A031CC2F8E7C
2016-01-14 18:31:14,371 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E0B95840A3B345E38C5093233C7FFD4B, all 1 inputs ready in: 00:00:00.012
2016-01-14 18:31:14,373 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:14,374 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@225c4dc4; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@59f5907b
2016-01-14 18:31:14,389 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:14,389 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:14,391 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796272350_0001_1_01_000000_0
2016-01-14 18:31:14,391 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796272350_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:14,393 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796272350_0001_1_01_000000_0
2016-01-14 18:31:14,393 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E0B95840A3B345E38C5093233C7FFD4B, mem on close (mb), free: 81, total: 281, max: 672
2016-01-14 18:31:14,394 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D7DF6DB1B8CC4837B300A031CC2F8E7C
2016-01-14 18:31:14,394 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:14,395 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:14,395 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:14,395 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:14,396 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:14 Completed running task attempt: attempt_1452796272350_0001_1_01_000000_0
2016-01-14 18:31:14,397 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796272350_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:14,399 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:14,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=66, FIRST_EVENT_RECEIVED=29, LAST_EVENT_RECEIVED=29][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796274372, Process_Duration=21, Process_End_Time=1452796274393, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:14,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E0B95840A3B345E38C5093233C7FFD4B, taskAttemptId=attempt_1452796272350_0001_1_01_000000_0, startTime=1452796274232, finishTime=1452796274399, timeTaken=167, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=66, FIRST_EVENT_RECEIVED=29, LAST_EVENT_RECEIVED=29, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796274372, Process_Duration=21, Process_End_Time=1452796274393, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:14,405 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796272350_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:14,405 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796272350_0001_1_01_000000_0
2016-01-14 18:31:14,406 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:14,408 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:14,408 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:14,408 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:14,411 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:TASK_FINISHED]: vertexName=E0B95840A3B345E38C5093233C7FFD4B, taskId=task_1452796272350_0001_1_01_000000, startTime=1452796274232, finishTime=1452796274406, timeTaken=174, status=SUCCEEDED, successfulAttemptID=attempt_1452796272350_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=66, FIRST_EVENT_RECEIVED=29, LAST_EVENT_RECEIVED=29, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796274372, Process_Duration=21, Process_End_Time=1452796274393, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:14,411 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796272350_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:14,411 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796272350_0001_00_000002
2016-01-14 18:31:14,412 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B] : 1
2016-01-14 18:31:14,413 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B]
2016-01-14 18:31:14,415 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:14,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:VERTEX_FINISHED]: vertexName=E0B95840A3B345E38C5093233C7FFD4B, vertexId=vertex_1452796272350_0001_1_01, initRequestedTime=1452796273484, initedTime=1452796273590, startRequestedTime=1452796273658, startedTime=1452796273658, finishTime=1452796274413, timeTaken=755, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=295174144, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=66, FIRST_EVENT_RECEIVED=29, LAST_EVENT_RECEIVED=29, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796274372, Process_Duration=21, Process_End_Time=1452796274393, Read_Duration=4, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796274232, firstTasksToStart=[ task_1452796272350_0001_1_01_000000 ], lastTaskFinishTime=1452796274399, lastTasksToFinish=[ task_1452796272350_0001_1_01_000000 ], minTaskDuration=167, maxTaskDuration=167, avgTaskDuration=167.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796272350_0001_1_01_000000 ], longestDurationTasks=[ task_1452796272350_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:14,416 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:14,417 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:14,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796272350_0001_00_000002, stoppedTime=1452796274417, exitStatus=0
2016-01-14 18:31:14,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:14,418 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:14,418 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:14,418 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796272350_0001_1
2016-01-14 18:31:14,420 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796272350_0001_1
2016-01-14 18:31:14,420 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 79BB2AACFB004469AB2DA45DE06D94BE for vertex: vertex_1452796272350_0001_1_01 [E0B95840A3B345E38C5093233C7FFD4B]
2016-01-14 18:31:14,425 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796272350_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:14,425 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796272350_0001_00_000002: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:14,426 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796272350_0001_00_000002, Exiting
2016-01-14 18:31:14,426 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796272350_0001_00_000002
2016-01-14 18:31:14,426 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796272350_0001_00_000002
2016-01-14 18:31:14,428 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796272350_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:14,428 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796272350_0001_00_000002 completed successfully
2016-01-14 18:31:14,437 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796272350_0001_1_00 [D7DF6DB1B8CC4837B300A031CC2F8E7C]
2016-01-14 18:31:14,445 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796272350_0001_1, queueSize=0
2016-01-14 18:31:14,445 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:14,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796272350_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796272350_0001_1, startTime=1452796273447, finishTime=1452796274437, timeTaken=990, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=44, MERGE_PHASE_TIME=66, FIRST_EVENT_RECEIVED=29, LAST_EVENT_RECEIVED=29, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592548463, Process_Duration=41, Process_End_Time=2905592548504, Read_Duration=7, Tuples_Read=40, Tuples_Written=40, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:14,449 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796272350_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:14,449 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796272350_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:14,450 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796272350_0001_00_000002
2016-01-14 18:31:14,450 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796272350_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:14 Completed Dag: dag_1452796272350_0001_1
2016-01-14 18:31:14,451 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796272350_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:14,451 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796272350_0001_1_post
2016-01-14 18:31:14,451 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:14,451 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:14,462 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_'field'_-1, applicationId=application_1452796272350_0001
2016-01-14 18:31:14,471 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:14,471 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:14,471 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:14,471 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:14,471 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:14,472 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:14,475 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:14,475 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:14,475 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:14,475 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:14,475 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:14,476 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:14,476 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:14,482 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7AB586BF693241759A6C18C9AE58595B/D038D61CBD674A9596604C802242D34D/.tez/application_1452796272350_0001
2016-01-14 18:31:14,483 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:02.332
2016-01-14 18:31:14,614 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_-1
2016-01-14 18:31:14,690 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:14,691 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:14,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.043
2016-01-14 18:31:14,810 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.074
2016-01-14 18:31:14,811 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:14,811 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:14,812 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:14,835 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:14,851 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'
2016-01-14 18:31:14,939 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:14,940 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:14,940 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'"]
2016-01-14 18:31:14,941 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:14,942 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:14,942 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:14,944 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/-1_'field'_'field'
2016-01-14 18:31:14,944 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:14,966 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:14,971 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:14,972 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:14,972 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:14,983 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF/.tez/application_1452796274971_0001 doesn't exist and is created
2016-01-14 18:31:15,001 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:15,002 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:15,003 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF/.tez/application_1452796274971_0001
2016-01-14 18:31:15,005 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796274971_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:15,022 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:15,024 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:15,026 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:15,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:15,027 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF/.tez/application_1452796274971_0001/localmode-log-dir
2016-01-14 18:31:15,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF/.tez/application_1452796274971_0001/localmode-log-dir/history.txt.appattempt_1452796274971_0001_000000, maxErrors=10
2016-01-14 18:31:15,028 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:15,029 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796274971_0001_000000, appSubmitTime=1452796275005, launchTime=1452796275005
2016-01-14 18:31:15,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:15,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:15,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:15,047 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:15,053 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59911
2016-01-14 18:31:15,055 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:15,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796274971_0001_000000, startTime=1452796275059
2016-01-14 18:31:15,059 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:15,060 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:15,102 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:15,102 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:15,102 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1452796274971_0001, dagName=[242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF] every/(1/1) ...atform/-1_'field'_'field'
2016-01-14 18:31:15,128 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF] every/(1/1) ...atform/-1_'field'_'field'
2016-01-14 18:31:15,132 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796274971_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF/.tez/application_1452796274971_0001/localmode-log-dir/dag_1452796274971_0001_1.dot
2016-01-14 18:31:15,133 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF/.tez/application_1452796274971_0001/localmode-log-dir/dag_1452796274971_0001_1-tez-dag.pb.txt
2016-01-14 18:31:15,170 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796274971_0001_1
2016-01-14 18:31:15,170 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:15,170 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF] every/(1/1) ...atform/-1_'field'_'field'
2016-01-14 18:31:15 Running Dag: dag_1452796274971_0001_1
2016-01-14 18:31:15,183 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796274971_0001_1, submitTime=1452796275128
2016-01-14 18:31:15,183 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:15,184 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F3A75A3762C1474CBCA3F82282A18DF7
2016-01-14 18:31:15,184 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:15,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:15,184 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796274971_0001_1, initTime=1452796275183
2016-01-14 18:31:15,186 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796274971_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:15,186 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:DAG_STARTED]: dagID=dag_1452796274971_0001_1, startTime=1452796275186
2016-01-14 18:31:15,187 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:15,187 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796274971_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:15,187 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D5B1AA04AC134808B679067F5750CF42 : {2AE87FD07CEC45D3909695B964BD1FD5={InputName=2AE87FD07CEC45D3909695B964BD1FD5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:15,186 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:15,187 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2AE87FD07CEC45D3909695B964BD1FD5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:15,188 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42]
2016-01-14 18:31:15,188 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42]
2016-01-14 18:31:15,188 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1452796274971_0001, dagName=[242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF] every/(1/1) ...atform/-1_'field'_'field'
2016-01-14 18:31:15,188 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42]
2016-01-14 18:31:15,189 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42]. Starting root input initializers: 1
2016-01-14 18:31:15,189 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:15,190 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7]
2016-01-14 18:31:15,192 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2AE87FD07CEC45D3909695B964BD1FD5 on vertex vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42]
2016-01-14 18:31:15,196 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:15,196 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7]
2016-01-14 18:31:15,197 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7]
2016-01-14 18:31:15,197 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7]
2016-01-14 18:31:15,197 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=DE7D8647F3F245808731B748DF4AA056, vertexId=vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:15,198 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=DE7D8647F3F245808731B748DF4AA056, vertexId=vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7]
2016-01-14 18:31:15,197 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:15,200 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2AE87FD07CEC45D3909695B964BD1FD5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:15,202 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:15,203 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:15,204 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=DE7D8647F3F245808731B748DF4AA056, vertexName=F3A75A3762C1474CBCA3F82282A18DF7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:15,204 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=DE7D8647F3F245808731B748DF4AA056, vertexId=vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7]
2016-01-14 18:31:15,206 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:15,207 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F3A75A3762C1474CBCA3F82282A18DF7, vertexId=vertex_1452796274971_0001_1_01, initRequestedTime=1452796275190, initedTime=1452796275206, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:15,207 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:15,208 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:15,210 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2AE87FD07CEC45D3909695B964BD1FD5 on vertex vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42]
2016-01-14 18:31:15,211 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:15,211 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2AE87FD07CEC45D3909695B964BD1FD5=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:15,213 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42] parallelism set to 1
2016-01-14 18:31:15,213 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:15,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796274971_0001_1_00, eventCount=1
2016-01-14 18:31:15,215 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:15,218 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5B1AA04AC134808B679067F5750CF42, vertexId=vertex_1452796274971_0001_1_00, initRequestedTime=1452796275187, initedTime=1452796275215, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:15,219 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42] with distanceFromRoot: 0
2016-01-14 18:31:15,219 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:15,219 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D5B1AA04AC134808B679067F5750CF42
2016-01-14 18:31:15,221 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42]
2016-01-14 18:31:15,221 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:15,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796274971_0001_1_00, startRequestedTime=1452796275219, startedTime=1452796275219
2016-01-14 18:31:15,223 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:15,225 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:15,226 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:TASK_STARTED]: vertexName=D5B1AA04AC134808B679067F5750CF42, taskId=task_1452796274971_0001_1_00_000000, scheduledTime=1452796275225, launchTime=1452796275225
2016-01-14 18:31:15,226 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796274971_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:15,226 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796274971_0001_1_00 for vertex: vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7] numStartedSources: 1 numSources: 1
2016-01-14 18:31:15,226 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7] with distanceFromRoot: 1
2016-01-14 18:31:15,226 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42]
2016-01-14 18:31:15,227 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D5B1AA04AC134808B679067F5750CF42 in vertex: F3A75A3762C1474CBCA3F82282A18DF7
2016-01-14 18:31:15,227 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F3A75A3762C1474CBCA3F82282A18DF7 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:15,227 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:15,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796274971_0001_1_01, startRequestedTime=1452796275227, startedTime=1452796275227
2016-01-14 18:31:15,228 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:15,228 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF] every/(1/1) ...atform/-1_'field'_'field', VertexName: D5B1AA04AC134808B679067F5750CF42, VertexParallelism: 1, TaskAttemptID:attempt_1452796274971_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2AE87FD07CEC45D3909695B964BD1FD5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F3A75A3762C1474CBCA3F82282A18DF7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:15,229 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796274971_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:15,229 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:15,230 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:15,232 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:15,234 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:15,235 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796274971_0001, with dag id: dag_1452796274971_0001_1
2016-01-14 18:31:15,235 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796274971_0001_00_000001
2016-01-14 18:31:15,236 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:15,242 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796274971_0001_00_000001
2016-01-14 18:31:15,242 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:15,243 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:15,243 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:15,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796274971_0001_00_000001, launchTime=1452796275243
2016-01-14 18:31:15,245 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796274971_0001_00_000001
2016-01-14 18:31:15,246 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796274971_0001_1_00_000000_0] to container: [container_1452796274971_0001_00_000001]
2016-01-14 18:31:15,246 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:15,246 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796274971_0001_00_000001 given task: attempt_1452796274971_0001_1_00_000000_0
2016-01-14 18:31:15,247 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796274971_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796274971_0001_1_00_000000_0
2016-01-14 18:31:15,247 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796274971_0001_1_00_000000_0] started. Is using containerId: [container_1452796274971_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:15 Starting to run new task attempt: attempt_1452796274971_0001_1_00_000000_0
2016-01-14 18:31:15,247 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5B1AA04AC134808B679067F5750CF42, taskAttemptId=attempt_1452796274971_0001_1_00_000000_0, startTime=1452796275246, containerId=container_1452796274971_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796274971_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:15,247 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796274971_0001_1_00_000000_0
2016-01-14 18:31:15,247 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796274971_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:15,247 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796274971_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:15,247 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:15,248 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:15,249 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:15,249 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:15,249 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:15,249 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:15,249 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:15,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF] every/(1/1) ...atform/-1_'field'_'field', VertexName: D5B1AA04AC134808B679067F5750CF42, VertexParallelism: 1, TaskAttemptID:attempt_1452796274971_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2AE87FD07CEC45D3909695B964BD1FD5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F3A75A3762C1474CBCA3F82282A18DF7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:15,250 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:15,251 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796274971_0001_1_00_000000_0
2016-01-14 18:31:15,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:15,252 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:15,252 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796274971_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:15,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2AE87FD07CEC45D3909695B964BD1FD5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:15,254 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F3A75A3762C1474CBCA3F82282A18DF7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:15,254 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:15,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:15,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:15,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F3A75A3762C1474CBCA3F82282A18DF7
2016-01-14 18:31:15,259 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:15,259 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F3A75A3762C1474CBCA3F82282A18DF7
2016-01-14 18:31:15,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F3A75A3762C1474CBCA3F82282A18DF7
2016-01-14 18:31:15,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2AE87FD07CEC45D3909695B964BD1FD5
2016-01-14 18:31:15,260 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2AE87FD07CEC45D3909695B964BD1FD5
2016-01-14 18:31:15,259 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:15,269 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5B1AA04AC134808B679067F5750CF42, ordinal: 0
2016-01-14 18:31:15,269 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5B1AA04AC134808B679067F5750CF42, mem on start (mb), free: 76, total: 281, max: 672
2016-01-14 18:31:15,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:15,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:15,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:15,271 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:15,271 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2AE87FD07CEC45D3909695B964BD1FD5
2016-01-14 18:31:15,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2AE87FD07CEC45D3909695B964BD1FD5
2016-01-14 18:31:15,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:15,272 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:15,272 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:15,273 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:15,273 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:15,273 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:15,274 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:15,274 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F3A75A3762C1474CBCA3F82282A18DF7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:15,274 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2AE87FD07CEC45D3909695B964BD1FD5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:15,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:15,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2AE87FD07CEC45D3909695B964BD1FD5 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:15,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:15,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2AE87FD07CEC45D3909695B964BD1FD5
2016-01-14 18:31:15,276 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:15,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2AE87FD07CEC45D3909695B964BD1FD5
2016-01-14 18:31:15,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:15,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:15,277 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796274971_0001_1_00_000000_0
2016-01-14 18:31:15,280 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:15,281 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@66587315
2016-01-14 18:31:15,283 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@66587315
2016-01-14 18:31:15,284 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:15,284 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:15,286 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:15,287 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2AE87FD07CEC45D3909695B964BD1FD5
2016-01-14 18:31:15,287 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 9CF3CBC5E5BF49289EE48FA7B1547A66
2016-01-14 18:31:15,288 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 9CF3CBC5E5BF49289EE48FA7B1547A66
2016-01-14 18:31:15,289 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@794193b6; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@4048cfeb; comparator=cascading.tuple.hadoop.util.TupleComparator@a8dab8e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:15,290 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:15,290 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:15,290 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:15,403 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:15,403 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:15,403 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:15,404 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:15,404 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:15,405 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:15,405 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:15,406 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5B1AA04AC134808B679067F5750CF42, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:15,409 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796274971_0001_1_00_000000_0
2016-01-14 18:31:15,409 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5B1AA04AC134808B679067F5750CF42, mem on close (mb), free: 73, total: 382, max: 672
2016-01-14 18:31:15,410 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:15,410 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:15,410 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:15,411 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:15,413 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:15 Completed running task attempt: attempt_1452796274971_0001_1_00_000000_0
2016-01-14 18:31:15,417 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796274971_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:15,418 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796274971_0001_1_00, eventCount=1
2016-01-14 18:31:15,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796275406, Process_Duration=3, Process_End_Time=1452796275409, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:15,419 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:15,419 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:15,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:15,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5B1AA04AC134808B679067F5750CF42, taskAttemptId=attempt_1452796274971_0001_1_00_000000_0, startTime=1452796275246, finishTime=1452796275418, timeTaken=172, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796275406, Process_Duration=3, Process_End_Time=1452796275409, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:15,421 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796274971_0001_00_000001
2016-01-14 18:31:15,421 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796274971_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:15,422 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:15,422 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796274971_0001_1_00_000000_0
2016-01-14 18:31:15,423 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:15,423 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:TASK_FINISHED]: vertexName=D5B1AA04AC134808B679067F5750CF42, taskId=task_1452796274971_0001_1_00_000000, startTime=1452796275246, finishTime=1452796275423, timeTaken=177, status=SUCCEEDED, successfulAttemptID=attempt_1452796274971_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796275406, Process_Duration=3, Process_End_Time=1452796275409, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:15,425 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796274971_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:15,425 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42] : 1
2016-01-14 18:31:15,426 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42]
2016-01-14 18:31:15,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:15,428 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:VERTEX_FINISHED]: vertexName=D5B1AA04AC134808B679067F5750CF42, vertexId=vertex_1452796274971_0001_1_00, initRequestedTime=1452796275187, initedTime=1452796275215, startRequestedTime=1452796275219, startedTime=1452796275219, finishTime=1452796275426, timeTaken=207, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796275406, Process_Duration=3, Process_End_Time=1452796275409, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796275246, firstTasksToStart=[ task_1452796274971_0001_1_00_000000 ], lastTaskFinishTime=1452796275418, lastTasksToFinish=[ task_1452796274971_0001_1_00_000000 ], minTaskDuration=172, maxTaskDuration=172, avgTaskDuration=172.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796274971_0001_1_00_000000 ], longestDurationTasks=[ task_1452796274971_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:15,428 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:15,428 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:15,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796274971_0001_00_000001, stoppedTime=1452796275428, exitStatus=0
2016-01-14 18:31:15,429 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:15,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:15,429 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7] attempt: attempt_1452796274971_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:15,429 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F3A75A3762C1474CBCA3F82282A18DF7
2016-01-14 18:31:15,430 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7]
2016-01-14 18:31:15,430 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:15,430 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796274971_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:15,430 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:15,430 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:TASK_STARTED]: vertexName=F3A75A3762C1474CBCA3F82282A18DF7, taskId=task_1452796274971_0001_1_01_000000, scheduledTime=1452796275430, launchTime=1452796275430
2016-01-14 18:31:15,431 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796274971_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:15,431 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7]
2016-01-14 18:31:15,431 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:15,432 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF] every/(1/1) ...atform/-1_'field'_'field', VertexName: F3A75A3762C1474CBCA3F82282A18DF7, VertexParallelism: 1, TaskAttemptID:attempt_1452796274971_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5B1AA04AC134808B679067F5750CF42, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DE7D8647F3F245808731B748DF4AA056, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:15,432 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796274971_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:15,433 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:15,433 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796274971_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:15,434 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796274971_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:15,434 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796274971_0001_00_000001, Exiting
2016-01-14 18:31:15,434 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796274971_0001_00_000001
2016-01-14 18:31:15,435 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796274971_0001_00_000001
2016-01-14 18:31:15,434 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796274971_0001_00_000002
2016-01-14 18:31:15,439 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:15,439 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796274971_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:15,439 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796274971_0001_00_000001 completed successfully
2016-01-14 18:31:15,439 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:15,442 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796274971_0001_00_000002
2016-01-14 18:31:15,443 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:15,445 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:15,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796274971_0001_00_000002, launchTime=1452796275445
2016-01-14 18:31:15,445 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:15,446 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796274971_0001_00_000002
2016-01-14 18:31:15,447 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796274971_0001_1_01_000000_0] to container: [container_1452796274971_0001_00_000002]
2016-01-14 18:31:15,447 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:15,447 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796274971_0001_00_000002 given task: attempt_1452796274971_0001_1_01_000000_0
2016-01-14 18:31:15,448 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796274971_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796274971_0001_1_01_000000_0
2016-01-14 18:31:15,448 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796274971_0001_1_01_000000_0] started. Is using containerId: [container_1452796274971_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:15,448 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:15,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F3A75A3762C1474CBCA3F82282A18DF7, taskAttemptId=attempt_1452796274971_0001_1_01_000000_0, startTime=1452796275448, containerId=container_1452796274971_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796274971_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:15 Starting to run new task attempt: attempt_1452796274971_0001_1_01_000000_0
2016-01-14 18:31:15,449 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796274971_0001_1_01_000000_0
2016-01-14 18:31:15,449 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:15,449 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:15,450 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:15,450 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:15,451 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:15,451 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:15,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF] every/(1/1) ...atform/-1_'field'_'field', VertexName: F3A75A3762C1474CBCA3F82282A18DF7, VertexParallelism: 1, TaskAttemptID:attempt_1452796274971_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5B1AA04AC134808B679067F5750CF42, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=DE7D8647F3F245808731B748DF4AA056, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:15,450 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796274971_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:15,452 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796274971_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:15,451 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:15,453 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796274971_0001_1_01_000000_0
2016-01-14 18:31:15,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:15,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:15,455 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796274971_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:15,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DE7D8647F3F245808731B748DF4AA056, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:15,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:15,455 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D5B1AA04AC134808B679067F5750CF42, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:15,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:15,457 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D5B1AA04AC134808B679067F5750CF42
2016-01-14 18:31:15,460 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:15,461 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:15,461 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D5B1AA04AC134808B679067F5750CF42
2016-01-14 18:31:15,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D5B1AA04AC134808B679067F5750CF42
2016-01-14 18:31:15,462 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:15,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:15,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DE7D8647F3F245808731B748DF4AA056
2016-01-14 18:31:15,467 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:15,467 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: DE7D8647F3F245808731B748DF4AA056
2016-01-14 18:31:15,480 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:15,480 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DE7D8647F3F245808731B748DF4AA056
2016-01-14 18:31:15,483 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F3A75A3762C1474CBCA3F82282A18DF7, ordinal: 1
2016-01-14 18:31:15,483 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F3A75A3762C1474CBCA3F82282A18DF7, mem on start (mb), free: 65, total: 382, max: 672
2016-01-14 18:31:15,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:15,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:15,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:15,484 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:15,485 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:15,485 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:15,485 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:15,486 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:15,486 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:15,487 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:15,487 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D5B1AA04AC134808B679067F5750CF42, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:15,487 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DE7D8647F3F245808731B748DF4AA056, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:15,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:15,488 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D5B1AA04AC134808B679067F5750CF42
2016-01-14 18:31:15,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D5B1AA04AC134808B679067F5750CF42 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:15,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:15,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:15,490 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:15,491 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:15,493 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D5B1AA04AC134808B679067F5750CF42 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:15,494 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:15,494 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:15,494 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:15,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D5B1AA04AC134808B679067F5750CF42
2016-01-14 18:31:15,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:15,500 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796274971_0001_1_01_000000_0
2016-01-14 18:31:15,504 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796274971_0001_1_00_000000_0_10009, runDuration: 160175, ]
2016-01-14 18:31:15,506 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:15,508 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796274971_0001_1_00_000000_0_10009] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796275508, TimeTaken=3, Rate=0.10 MB/s
2016-01-14 18:31:15,509 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D5B1AA04AC134808B679067F5750CF42
2016-01-14 18:31:15,509 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:15,510 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D5B1AA04AC134808B679067F5750CF42] #2 in 5ms
2016-01-14 18:31:15,510 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D5B1AA04AC134808B679067F5750CF42] #2, status:false, isInterrupted:false
2016-01-14 18:31:15,511 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:15,509 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 9CF3CBC5E5BF49289EE48FA7B1547A66
2016-01-14 18:31:15,511 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:15,512 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:15,512 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:15,513 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'"], id: DE7D8647F3F245808731B748DF4AA056
2016-01-14 18:31:15,513 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_'field'"]
2016-01-14 18:31:15,513 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 9CF3CBC5E5BF49289EE48FA7B1547A66, for 1 inputs
2016-01-14 18:31:15,513 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:15,514 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D5B1AA04AC134808B679067F5750CF42
2016-01-14 18:31:15,516 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:15,516 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F3A75A3762C1474CBCA3F82282A18DF7, all 1 inputs ready in: 00:00:00.002
2016-01-14 18:31:15,517 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@64a3b9dd; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@38ff8fc9
2016-01-14 18:31:15,521 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:15,521 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:15,522 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796274971_0001_1_01_000000_0
2016-01-14 18:31:15,522 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796274971_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:15,523 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796274971_0001_1_01_000000_0
2016-01-14 18:31:15,523 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F3A75A3762C1474CBCA3F82282A18DF7, mem on close (mb), free: 61, total: 382, max: 672
2016-01-14 18:31:15,523 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D5B1AA04AC134808B679067F5750CF42
2016-01-14 18:31:15,524 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:15,524 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:15,524 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:15,524 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:15,524 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:15 Completed running task attempt: attempt_1452796274971_0001_1_01_000000_0
2016-01-14 18:31:15,525 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796274971_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:15,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796275516, Process_Duration=7, Process_End_Time=1452796275523, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:15,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:15,527 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:15,527 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:15,528 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796274971_0001_00_000002
2016-01-14 18:31:15,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F3A75A3762C1474CBCA3F82282A18DF7, taskAttemptId=attempt_1452796274971_0001_1_01_000000_0, startTime=1452796275448, finishTime=1452796275526, timeTaken=78, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796275516, Process_Duration=7, Process_End_Time=1452796275523, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:15,529 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796274971_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:15,529 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:15,529 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796274971_0001_1_01_000000_0
2016-01-14 18:31:15,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:15,536 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:TASK_FINISHED]: vertexName=F3A75A3762C1474CBCA3F82282A18DF7, taskId=task_1452796274971_0001_1_01_000000, startTime=1452796275448, finishTime=1452796275530, timeTaken=82, status=SUCCEEDED, successfulAttemptID=attempt_1452796274971_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796275516, Process_Duration=7, Process_End_Time=1452796275523, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:15,536 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796274971_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:15,536 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7] : 1
2016-01-14 18:31:15,536 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7]
2016-01-14 18:31:15,538 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:15,540 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:VERTEX_FINISHED]: vertexName=F3A75A3762C1474CBCA3F82282A18DF7, vertexId=vertex_1452796274971_0001_1_01, initRequestedTime=1452796275190, initedTime=1452796275206, startRequestedTime=1452796275227, startedTime=1452796275227, finishTime=1452796275537, timeTaken=310, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=400556032, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796275516, Process_Duration=7, Process_End_Time=1452796275523, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796275448, firstTasksToStart=[ task_1452796274971_0001_1_01_000000 ], lastTaskFinishTime=1452796275526, lastTasksToFinish=[ task_1452796274971_0001_1_01_000000 ], minTaskDuration=78, maxTaskDuration=78, avgTaskDuration=78.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796274971_0001_1_01_000000 ], longestDurationTasks=[ task_1452796274971_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:15,540 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:15,540 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:15,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796274971_0001_00_000002, stoppedTime=1452796275541, exitStatus=0
2016-01-14 18:31:15,542 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:15,542 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796274971_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:15,542 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796274971_0001_00_000002: 14 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:15,542 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796274971_0001_00_000002, Exiting
2016-01-14 18:31:15,542 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796274971_0001_00_000002
2016-01-14 18:31:15,542 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796274971_0001_00_000002
2016-01-14 18:31:15,543 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796274971_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:15,543 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796274971_0001_00_000002 completed successfully
2016-01-14 18:31:15,543 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:15,544 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:15,544 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796274971_0001_1
2016-01-14 18:31:15,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796274971_0001_1
2016-01-14 18:31:15,544 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: DE7D8647F3F245808731B748DF4AA056 for vertex: vertex_1452796274971_0001_1_01 [F3A75A3762C1474CBCA3F82282A18DF7]
2016-01-14 18:31:15,556 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796274971_0001_1_00 [D5B1AA04AC134808B679067F5750CF42]
2016-01-14 18:31:15,559 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796274971_0001_1, queueSize=0
2016-01-14 18:31:15,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:15,562 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796274971_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796274971_0001_1, startTime=1452796275186, finishTime=1452796275556, timeTaken=370, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=801112064, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=18, MERGE_PHASE_TIME=22, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592550922, Process_Duration=10, Process_End_Time=2905592550932, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:15,562 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796274971_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:15,562 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796274971_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:15,563 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796274971_0001_00_000002
2016-01-14 18:31:15,566 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796274971_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:15 Completed Dag: dag_1452796274971_0001_1
2016-01-14 18:31:15,568 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796274971_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:15,569 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796274971_0001_1_post
2016-01-14 18:31:15,569 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:15,571 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:15,574 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_'field'_'field', applicationId=application_1452796274971_0001
2016-01-14 18:31:15,578 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:15,578 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:15,578 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:15,579 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:15,579 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:15,579 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:15,580 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:15,583 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:15,583 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:15,583 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:15,585 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:15,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:15,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:15,594 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/242997128A7144CEBEF54806291DD1C2/A07B8EDB7BB24EC78C66A9C242EBC6BF/.tez/application_1452796274971_0001
2016-01-14 18:31:15,595 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.656
2016-01-14 18:31:15,708 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_'field'
2016-01-14 18:31:15,744 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:15,744 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:15,807 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.030
2016-01-14 18:31:15,825 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.048
2016-01-14 18:31:15,826 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:15,826 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:15,827 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:15,858 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:15,869 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS
2016-01-14 18:31:15,936 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:15,937 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:15,937 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS"]
2016-01-14 18:31:15,937 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:15,938 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:15,938 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:15,940 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/-1_'field'_RESULTS
2016-01-14 18:31:15,940 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:15,958 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:15,963 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:15,964 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:15,964 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:15,974 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696/.tez/application_1452796275963_0001 doesn't exist and is created
2016-01-14 18:31:15,989 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:15,990 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:15,990 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696/.tez/application_1452796275963_0001
2016-01-14 18:31:15,993 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796275963_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:16,007 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:16,008 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:16,011 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:16,011 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:16,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696/.tez/application_1452796275963_0001/localmode-log-dir
2016-01-14 18:31:16,012 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696/.tez/application_1452796275963_0001/localmode-log-dir/history.txt.appattempt_1452796275963_0001_000000, maxErrors=10
2016-01-14 18:31:16,012 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:16,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796275963_0001_000000, appSubmitTime=1452796275992, launchTime=1452796275992
2016-01-14 18:31:16,018 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:16,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:16,023 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:39854
2016-01-14 18:31:16,027 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:16,028 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:16,030 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:16,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796275963_0001_000000, startTime=1452796276032
2016-01-14 18:31:16,032 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:16,032 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:16,090 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:16,090 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:16,090 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1452796275963_0001, dagName=[7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696] every/(1/1) ...atform/-1_'field'_RESULTS
2016-01-14 18:31:16,113 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696] every/(1/1) ...atform/-1_'field'_RESULTS
2016-01-14 18:31:16,115 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796275963_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696/.tez/application_1452796275963_0001/localmode-log-dir/dag_1452796275963_0001_1.dot
2016-01-14 18:31:16,118 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696/.tez/application_1452796275963_0001/localmode-log-dir/dag_1452796275963_0001_1-tez-dag.pb.txt
2016-01-14 18:31:16,154 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796275963_0001_1
2016-01-14 18:31:16,154 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:16,154 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696] every/(1/1) ...atform/-1_'field'_RESULTS
2016-01-14 18:31:16 Running Dag: dag_1452796275963_0001_1
2016-01-14 18:31:16,167 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796275963_0001_1, submitTime=1452796276113
2016-01-14 18:31:16,167 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:16,168 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A295114EFEDD4D7F9BFE8D1A42A12AFB
2016-01-14 18:31:16,169 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:16,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:16,169 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796275963_0001_1, initTime=1452796276167
2016-01-14 18:31:16,170 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796275963_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:16,171 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:DAG_STARTED]: dagID=dag_1452796275963_0001_1, startTime=1452796276171
2016-01-14 18:31:16,171 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:16,172 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1452796275963_0001, dagName=[7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696] every/(1/1) ...atform/-1_'field'_RESULTS
2016-01-14 18:31:16,171 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:16,172 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796275963_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:16,172 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CA718C9070794BEAAD817983E929C2AA : {57362E2B12E242F1A0FA85FEB9D9093F={InputName=57362E2B12E242F1A0FA85FEB9D9093F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:16,172 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 57362E2B12E242F1A0FA85FEB9D9093F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:16,172 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA]
2016-01-14 18:31:16,173 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA]
2016-01-14 18:31:16,173 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA]
2016-01-14 18:31:16,173 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA]. Starting root input initializers: 1
2016-01-14 18:31:16,174 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:16,174 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB]
2016-01-14 18:31:16,175 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 57362E2B12E242F1A0FA85FEB9D9093F on vertex vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA]
2016-01-14 18:31:16,179 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:16,179 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB]
2016-01-14 18:31:16,179 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB]
2016-01-14 18:31:16,179 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB]
2016-01-14 18:31:16,180 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=4BFFF81C213B48DDA40541010A051ECE, vertexId=vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:16,180 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=4BFFF81C213B48DDA40541010A051ECE, vertexId=vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB]
2016-01-14 18:31:16,181 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:16,183 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 57362E2B12E242F1A0FA85FEB9D9093F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:16,185 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:16,186 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:16,187 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=4BFFF81C213B48DDA40541010A051ECE, vertexName=A295114EFEDD4D7F9BFE8D1A42A12AFB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:16,188 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=4BFFF81C213B48DDA40541010A051ECE, vertexId=vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB]
2016-01-14 18:31:16,189 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:16,190 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 57362E2B12E242F1A0FA85FEB9D9093F on vertex vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA]
2016-01-14 18:31:16,191 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:16,191 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A295114EFEDD4D7F9BFE8D1A42A12AFB, vertexId=vertex_1452796275963_0001_1_01, initRequestedTime=1452796276174, initedTime=1452796276190, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:16,191 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:16,191 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:16,192 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {57362E2B12E242F1A0FA85FEB9D9093F=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:16,192 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA] parallelism set to 1
2016-01-14 18:31:16,194 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:16,195 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796275963_0001_1_00, eventCount=1
2016-01-14 18:31:16,197 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:16,198 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CA718C9070794BEAAD817983E929C2AA, vertexId=vertex_1452796275963_0001_1_00, initRequestedTime=1452796276172, initedTime=1452796276196, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:16,198 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA] with distanceFromRoot: 0
2016-01-14 18:31:16,198 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:16,198 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CA718C9070794BEAAD817983E929C2AA
2016-01-14 18:31:16,200 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA]
2016-01-14 18:31:16,200 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796275963_0001_1_00, startRequestedTime=1452796276198, startedTime=1452796276198
2016-01-14 18:31:16,201 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:16,201 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:16,202 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:16,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:TASK_STARTED]: vertexName=CA718C9070794BEAAD817983E929C2AA, taskId=task_1452796275963_0001_1_00_000000, scheduledTime=1452796276201, launchTime=1452796276201
2016-01-14 18:31:16,203 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796275963_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:16,203 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796275963_0001_1_00 for vertex: vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB] numStartedSources: 1 numSources: 1
2016-01-14 18:31:16,203 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB] with distanceFromRoot: 1
2016-01-14 18:31:16,204 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA]
2016-01-14 18:31:16,205 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CA718C9070794BEAAD817983E929C2AA in vertex: A295114EFEDD4D7F9BFE8D1A42A12AFB
2016-01-14 18:31:16,205 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A295114EFEDD4D7F9BFE8D1A42A12AFB with 1 source tasks and 1 pending tasks
2016-01-14 18:31:16,206 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:16,206 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796275963_0001_1_01, startRequestedTime=1452796276205, startedTime=1452796276205
2016-01-14 18:31:16,206 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:16,207 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: CA718C9070794BEAAD817983E929C2AA, VertexParallelism: 1, TaskAttemptID:attempt_1452796275963_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=57362E2B12E242F1A0FA85FEB9D9093F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A295114EFEDD4D7F9BFE8D1A42A12AFB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:16,207 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796275963_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:16,208 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796275963_0001, with dag id: dag_1452796275963_0001_1
2016-01-14 18:31:16,210 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:16,212 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:16,214 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:16,215 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:16,221 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796275963_0001_00_000001
2016-01-14 18:31:16,221 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:16,223 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796275963_0001_00_000001
2016-01-14 18:31:16,223 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:16,224 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:16,224 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796275963_0001_00_000001, launchTime=1452796276224
2016-01-14 18:31:16,225 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:16,225 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796275963_0001_00_000001
2016-01-14 18:31:16,227 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796275963_0001_1_00_000000_0] to container: [container_1452796275963_0001_00_000001]
2016-01-14 18:31:16,227 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:16,228 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796275963_0001_00_000001 given task: attempt_1452796275963_0001_1_00_000000_0
2016-01-14 18:31:16,228 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796275963_0001_1_00_000000_0] started. Is using containerId: [container_1452796275963_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:16,229 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796275963_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796275963_0001_1_00_000000_0
2016-01-14 18:31:16,229 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:16 Starting to run new task attempt: attempt_1452796275963_0001_1_00_000000_0
2016-01-14 18:31:16,229 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796275963_0001_1_00_000000_0
2016-01-14 18:31:16,230 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:16,230 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:16,230 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:16,230 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:16,231 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:16,231 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:16,231 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: CA718C9070794BEAAD817983E929C2AA, VertexParallelism: 1, TaskAttemptID:attempt_1452796275963_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=57362E2B12E242F1A0FA85FEB9D9093F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A295114EFEDD4D7F9BFE8D1A42A12AFB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:16,231 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:16,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CA718C9070794BEAAD817983E929C2AA, taskAttemptId=attempt_1452796275963_0001_1_00_000000_0, startTime=1452796276228, containerId=container_1452796275963_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796275963_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:16,232 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796275963_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:16,232 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796275963_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:16,233 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796275963_0001_1_00_000000_0
2016-01-14 18:31:16,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:16,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:16,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:16,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A295114EFEDD4D7F9BFE8D1A42A12AFB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:16,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=57362E2B12E242F1A0FA85FEB9D9093F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:16,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:16,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A295114EFEDD4D7F9BFE8D1A42A12AFB
2016-01-14 18:31:16,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:16,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 57362E2B12E242F1A0FA85FEB9D9093F
2016-01-14 18:31:16,236 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 57362E2B12E242F1A0FA85FEB9D9093F
2016-01-14 18:31:16,237 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:16,237 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A295114EFEDD4D7F9BFE8D1A42A12AFB
2016-01-14 18:31:16,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A295114EFEDD4D7F9BFE8D1A42A12AFB
2016-01-14 18:31:16,241 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:16,242 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796275963_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:16,242 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:16,243 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 57362E2B12E242F1A0FA85FEB9D9093F
2016-01-14 18:31:16,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 57362E2B12E242F1A0FA85FEB9D9093F
2016-01-14 18:31:16,250 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CA718C9070794BEAAD817983E929C2AA, ordinal: 0
2016-01-14 18:31:16,250 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA718C9070794BEAAD817983E929C2AA, mem on start (mb), free: 164, total: 490, max: 672
2016-01-14 18:31:16,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:16,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:16,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:16,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:16,250 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:16,251 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:16,251 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:16,251 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:16,251 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:16,252 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:16,252 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 57362E2B12E242F1A0FA85FEB9D9093F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:16,252 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A295114EFEDD4D7F9BFE8D1A42A12AFB, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:16,252 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:16,252 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 57362E2B12E242F1A0FA85FEB9D9093F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:16,252 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 57362E2B12E242F1A0FA85FEB9D9093F
2016-01-14 18:31:16,253 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:16,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 57362E2B12E242F1A0FA85FEB9D9093F
2016-01-14 18:31:16,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:16,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:16,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:16,254 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796275963_0001_1_00_000000_0
2016-01-14 18:31:16,256 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:16,256 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:16,257 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 57362E2B12E242F1A0FA85FEB9D9093F
2016-01-14 18:31:16,257 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 978CC56687DD4E54AD30449CCA228B40
2016-01-14 18:31:16,258 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1f9b2a40
2016-01-14 18:31:16,258 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 978CC56687DD4E54AD30449CCA228B40
2016-01-14 18:31:16,259 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@568b8a74; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@632ef20f; comparator=cascading.tuple.hadoop.util.TupleComparator@1f15e9a8; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:16,260 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1f9b2a40
2016-01-14 18:31:16,261 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:16,261 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:16,261 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:16,261 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:16,262 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:16,343 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:16,344 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:16,344 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:16,344 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:16,344 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:16,345 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:16,345 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:16,346 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CA718C9070794BEAAD817983E929C2AA, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:16,348 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796275963_0001_1_00_000000_0
2016-01-14 18:31:16,349 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CA718C9070794BEAAD817983E929C2AA, mem on close (mb), free: 62, total: 490, max: 672
2016-01-14 18:31:16,349 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:16,349 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:16,349 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:16,350 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:16,353 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:16 Completed running task attempt: attempt_1452796275963_0001_1_00_000000_0
2016-01-14 18:31:16,357 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796275963_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:16,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796275963_0001_1_00, eventCount=1
2016-01-14 18:31:16,358 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796276346, Process_Duration=2, Process_End_Time=1452796276348, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:16,359 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:16,359 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:16,359 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796275963_0001_00_000001
2016-01-14 18:31:16,360 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:16,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CA718C9070794BEAAD817983E929C2AA, taskAttemptId=attempt_1452796275963_0001_1_00_000000_0, startTime=1452796276228, finishTime=1452796276360, timeTaken=132, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796276346, Process_Duration=2, Process_End_Time=1452796276348, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:16,361 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796275963_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:16,362 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796275963_0001_1_00_000000_0
2016-01-14 18:31:16,362 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:16,363 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:TASK_FINISHED]: vertexName=CA718C9070794BEAAD817983E929C2AA, taskId=task_1452796275963_0001_1_00_000000, startTime=1452796276228, finishTime=1452796276362, timeTaken=134, status=SUCCEEDED, successfulAttemptID=attempt_1452796275963_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796276346, Process_Duration=2, Process_End_Time=1452796276348, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:16,363 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796275963_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:16,363 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA] : 1
2016-01-14 18:31:16,363 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA]
2016-01-14 18:31:16,364 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:16,365 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:16,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:VERTEX_FINISHED]: vertexName=CA718C9070794BEAAD817983E929C2AA, vertexId=vertex_1452796275963_0001_1_00, initRequestedTime=1452796276172, initedTime=1452796276196, startRequestedTime=1452796276198, startedTime=1452796276198, finishTime=1452796276363, timeTaken=165, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796276346, Process_Duration=2, Process_End_Time=1452796276348, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796276228, firstTasksToStart=[ task_1452796275963_0001_1_00_000000 ], lastTaskFinishTime=1452796276360, lastTasksToFinish=[ task_1452796275963_0001_1_00_000000 ], minTaskDuration=132, maxTaskDuration=132, avgTaskDuration=132.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796275963_0001_1_00_000000 ], longestDurationTasks=[ task_1452796275963_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:16,367 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:16,367 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB] attempt: attempt_1452796275963_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:16,368 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A295114EFEDD4D7F9BFE8D1A42A12AFB
2016-01-14 18:31:16,368 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB]
2016-01-14 18:31:16,368 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:16,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796275963_0001_00_000001, stoppedTime=1452796276368, exitStatus=0
2016-01-14 18:31:16,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:16,369 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:16,369 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:16,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:16,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:TASK_STARTED]: vertexName=A295114EFEDD4D7F9BFE8D1A42A12AFB, taskId=task_1452796275963_0001_1_01_000000, scheduledTime=1452796276370, launchTime=1452796276370
2016-01-14 18:31:16,370 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796275963_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:16,371 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796275963_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:16,371 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB]
2016-01-14 18:31:16,371 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:16,371 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: A295114EFEDD4D7F9BFE8D1A42A12AFB, VertexParallelism: 1, TaskAttemptID:attempt_1452796275963_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CA718C9070794BEAAD817983E929C2AA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4BFFF81C213B48DDA40541010A051ECE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:16,372 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796275963_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:16,372 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796275963_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:16,372 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:16,375 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796275963_0001_00_000001: 14 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:16,376 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796275963_0001_00_000002
2016-01-14 18:31:16,376 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796275963_0001_00_000001, Exiting
2016-01-14 18:31:16,376 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796275963_0001_00_000001
2016-01-14 18:31:16,376 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796275963_0001_00_000001
2016-01-14 18:31:16,376 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:16,378 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796275963_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:16,379 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796275963_0001_00_000001 completed successfully
2016-01-14 18:31:16,380 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796275963_0001_00_000002
2016-01-14 18:31:16,381 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:16,382 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:16,384 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:16,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796275963_0001_00_000002, launchTime=1452796276384
2016-01-14 18:31:16,384 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:16,385 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796275963_0001_00_000002
2016-01-14 18:31:16,386 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796275963_0001_1_01_000000_0] to container: [container_1452796275963_0001_00_000002]
2016-01-14 18:31:16,386 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:16,387 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796275963_0001_00_000002 given task: attempt_1452796275963_0001_1_01_000000_0
2016-01-14 18:31:16,387 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796275963_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796275963_0001_1_01_000000_0
2016-01-14 18:31:16,387 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796275963_0001_1_01_000000_0] started. Is using containerId: [container_1452796275963_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:16 Starting to run new task attempt: attempt_1452796275963_0001_1_01_000000_0
2016-01-14 18:31:16,387 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:16,387 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796275963_0001_1_01_000000_0
2016-01-14 18:31:16,388 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:16,388 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:16,388 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A295114EFEDD4D7F9BFE8D1A42A12AFB, taskAttemptId=attempt_1452796275963_0001_1_01_000000_0, startTime=1452796276387, containerId=container_1452796275963_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796275963_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:16,388 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:16,388 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796275963_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:16,388 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:16,388 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796275963_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:16,388 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:16,389 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:16,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696] every/(1/1) ...atform/-1_'field'_RESULTS, VertexName: A295114EFEDD4D7F9BFE8D1A42A12AFB, VertexParallelism: 1, TaskAttemptID:attempt_1452796275963_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CA718C9070794BEAAD817983E929C2AA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4BFFF81C213B48DDA40541010A051ECE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:16,389 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:16,390 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796275963_0001_1_01_000000_0
2016-01-14 18:31:16,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:16,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:16,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:16,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=4BFFF81C213B48DDA40541010A051ECE, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:16,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:16,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 4BFFF81C213B48DDA40541010A051ECE
2016-01-14 18:31:16,393 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:16,394 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 4BFFF81C213B48DDA40541010A051ECE
2016-01-14 18:31:16,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CA718C9070794BEAAD817983E929C2AA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:16,401 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:16,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:16,401 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796275963_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:16,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CA718C9070794BEAAD817983E929C2AA
2016-01-14 18:31:16,408 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:16,408 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:16,409 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CA718C9070794BEAAD817983E929C2AA
2016-01-14 18:31:16,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CA718C9070794BEAAD817983E929C2AA
2016-01-14 18:31:16,414 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A295114EFEDD4D7F9BFE8D1A42A12AFB, ordinal: 1
2016-01-14 18:31:16,414 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A295114EFEDD4D7F9BFE8D1A42A12AFB, mem on start (mb), free: 54, total: 490, max: 672
2016-01-14 18:31:16,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:16,414 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:16,415 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:16,415 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:16,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 4BFFF81C213B48DDA40541010A051ECE
2016-01-14 18:31:16,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:16,416 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:16,416 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:16,416 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:16,416 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:16,416 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:16,417 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:16,417 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 4BFFF81C213B48DDA40541010A051ECE, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:16,417 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CA718C9070794BEAAD817983E929C2AA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:16,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:16,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CA718C9070794BEAAD817983E929C2AA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:16,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:16,417 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:16,418 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CA718C9070794BEAAD817983E929C2AA
2016-01-14 18:31:16,419 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:16,419 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:16,420 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CA718C9070794BEAAD817983E929C2AA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:16,420 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:16,420 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:16,420 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:16,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CA718C9070794BEAAD817983E929C2AA
2016-01-14 18:31:16,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:16,422 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796275963_0001_1_01_000000_0
2016-01-14 18:31:16,422 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796275963_0001_1_00_000000_0_10014, runDuration: 120623, ]
2016-01-14 18:31:16,425 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796275963_0001_1_00_000000_0_10014] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796276425, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:16,426 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CA718C9070794BEAAD817983E929C2AA
2016-01-14 18:31:16,426 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:16,426 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [CA718C9070794BEAAD817983E929C2AA] #3 in 2ms
2016-01-14 18:31:16,427 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CA718C9070794BEAAD817983E929C2AA] #3, status:false, isInterrupted:false
2016-01-14 18:31:16,428 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:16,428 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:16,452 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:16,452 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:16,452 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 978CC56687DD4E54AD30449CCA228B40
2016-01-14 18:31:16,453 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS"], id: 4BFFF81C213B48DDA40541010A051ECE
2016-01-14 18:31:16,453 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:16,453 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_RESULTS"]
2016-01-14 18:31:16,454 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 978CC56687DD4E54AD30449CCA228B40, for 1 inputs
2016-01-14 18:31:16,455 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:16,455 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CA718C9070794BEAAD817983E929C2AA
2016-01-14 18:31:16,455 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:16,456 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A295114EFEDD4D7F9BFE8D1A42A12AFB, all 1 inputs ready in: 00:00:00.002
2016-01-14 18:31:16,457 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4de4f698; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@72f99d1b
2016-01-14 18:31:16,460 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:16,461 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:16,462 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796275963_0001_1_01_000000_0
2016-01-14 18:31:16,462 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796275963_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:16,463 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796275963_0001_1_01_000000_0
2016-01-14 18:31:16,463 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A295114EFEDD4D7F9BFE8D1A42A12AFB, mem on close (mb), free: 257, total: 485, max: 672
2016-01-14 18:31:16,463 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CA718C9070794BEAAD817983E929C2AA
2016-01-14 18:31:16,464 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:16,464 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:16,464 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:16,464 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:16,464 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:16 Completed running task attempt: attempt_1452796275963_0001_1_01_000000_0
2016-01-14 18:31:16,465 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796275963_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:16,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=23, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=36, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796276456, Process_Duration=7, Process_End_Time=1452796276463, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:16,467 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:16,467 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:16,467 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A295114EFEDD4D7F9BFE8D1A42A12AFB, taskAttemptId=attempt_1452796275963_0001_1_01_000000_0, startTime=1452796276387, finishTime=1452796276466, timeTaken=79, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=23, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=36, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796276456, Process_Duration=7, Process_End_Time=1452796276463, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:16,468 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796275963_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:16,468 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796275963_0001_1_01_000000_0
2016-01-14 18:31:16,468 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:16,467 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796275963_0001_00_000002
2016-01-14 18:31:16,468 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:16,469 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:TASK_FINISHED]: vertexName=A295114EFEDD4D7F9BFE8D1A42A12AFB, taskId=task_1452796275963_0001_1_01_000000, startTime=1452796276387, finishTime=1452796276468, timeTaken=81, status=SUCCEEDED, successfulAttemptID=attempt_1452796275963_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=23, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=36, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796276456, Process_Duration=7, Process_End_Time=1452796276463, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:16,470 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796275963_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:16,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:16,471 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB] : 1
2016-01-14 18:31:16,471 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB]
2016-01-14 18:31:16,473 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:16,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:VERTEX_FINISHED]: vertexName=A295114EFEDD4D7F9BFE8D1A42A12AFB, vertexId=vertex_1452796275963_0001_1_01, initRequestedTime=1452796276174, initedTime=1452796276190, startRequestedTime=1452796276205, startedTime=1452796276205, finishTime=1452796276471, timeTaken=266, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=23, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=509083648, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=36, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796276456, Process_Duration=7, Process_End_Time=1452796276463, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796276387, firstTasksToStart=[ task_1452796275963_0001_1_01_000000 ], lastTaskFinishTime=1452796276466, lastTasksToFinish=[ task_1452796275963_0001_1_01_000000 ], minTaskDuration=79, maxTaskDuration=79, avgTaskDuration=79.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796275963_0001_1_01_000000 ], longestDurationTasks=[ task_1452796275963_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:16,473 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:16,474 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:16,475 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796275963_0001_00_000002, stoppedTime=1452796276475, exitStatus=0
2016-01-14 18:31:16,475 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:16,475 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:16,476 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:16,476 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796275963_0001_1
2016-01-14 18:31:16,476 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796275963_0001_1
2016-01-14 18:31:16,476 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 4BFFF81C213B48DDA40541010A051ECE for vertex: vertex_1452796275963_0001_1_01 [A295114EFEDD4D7F9BFE8D1A42A12AFB]
2016-01-14 18:31:16,481 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796275963_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:16,482 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796275963_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:16,482 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796275963_0001_00_000002, Exiting
2016-01-14 18:31:16,482 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796275963_0001_00_000002
2016-01-14 18:31:16,482 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796275963_0001_00_000002
2016-01-14 18:31:16,484 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796275963_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:16,485 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796275963_0001_00_000002 completed successfully
2016-01-14 18:31:16,485 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796275963_0001_1_00 [CA718C9070794BEAAD817983E929C2AA]
2016-01-14 18:31:16,488 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796275963_0001_1, queueSize=0
2016-01-14 18:31:16,489 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:16,489 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796275963_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796275963_0001_1, startTime=1452796276171, finishTime=1452796276486, timeTaken=315, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=23, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1023410176, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=36, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592552802, Process_Duration=9, Process_End_Time=2905592552811, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:16,489 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796275963_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:16,489 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796275963_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:16,489 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796275963_0001_00_000002
2016-01-14 18:31:16,490 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796275963_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:16 Completed Dag: dag_1452796275963_0001_1
2016-01-14 18:31:16,490 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796275963_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:16,490 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796275963_0001_1_post
2016-01-14 18:31:16,490 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:16,490 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:16,498 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_'field'_RESULTS, applicationId=application_1452796275963_0001
2016-01-14 18:31:16,502 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:16,502 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:16,502 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:16,502 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:16,502 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:16,503 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:16,504 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:16,505 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:16,505 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:16,505 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:16,505 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:16,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:16,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:16,515 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7F7AAD6895FE4A00AFD6D9A37FEBF67F/FC673855F0BF4C35B5267C8C58851696/.tez/application_1452796275963_0001
2016-01-14 18:31:16,516 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.579
2016-01-14 18:31:16,607 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_RESULTS
2016-01-14 18:31:16,635 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:16,635 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:16,689 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-01-14 18:31:16,698 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.032
2016-01-14 18:31:16,699 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:16,699 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:16,699 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:16,722 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:16,732 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL
2016-01-14 18:31:16,799 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:16,799 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:16,799 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL"]
2016-01-14 18:31:16,800 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:16,800 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:16,800 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:16,801 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/-1_'field'_ALL
2016-01-14 18:31:16,801 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:16,824 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:16,829 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:16,829 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:16,829 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:16,840 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770/.tez/application_1452796276829_0001 doesn't exist and is created
2016-01-14 18:31:16,855 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:16,856 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:16,856 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770/.tez/application_1452796276829_0001
2016-01-14 18:31:16,858 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796276829_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:16,871 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:16,872 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:16,874 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:16,874 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:16,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770/.tez/application_1452796276829_0001/localmode-log-dir
2016-01-14 18:31:16,875 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770/.tez/application_1452796276829_0001/localmode-log-dir/history.txt.appattempt_1452796276829_0001_000000, maxErrors=10
2016-01-14 18:31:16,875 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:16,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796276829_0001_000000, appSubmitTime=1452796276858, launchTime=1452796276858
2016-01-14 18:31:16,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:16,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:16,884 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:39356
2016-01-14 18:31:16,885 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:16,885 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:16,888 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:16,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796276829_0001_000000, startTime=1452796276890
2016-01-14 18:31:16,890 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:16,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:16,956 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:16,956 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:16,956 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1452796276829_0001, dagName=[346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-01-14 18:31:16,977 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-01-14 18:31:16,980 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796276829_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770/.tez/application_1452796276829_0001/localmode-log-dir/dag_1452796276829_0001_1.dot
2016-01-14 18:31:16,980 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770/.tez/application_1452796276829_0001/localmode-log-dir/dag_1452796276829_0001_1-tez-dag.pb.txt
2016-01-14 18:31:17,010 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796276829_0001_1
2016-01-14 18:31:17,010 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:17,010 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-01-14 18:31:17 Running Dag: dag_1452796276829_0001_1
2016-01-14 18:31:17,020 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:17,021 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796276829_0001_1, submitTime=1452796276977
2016-01-14 18:31:17,022 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BC8C1C9C2B8B493E8129604F932CFB03
2016-01-14 18:31:17,022 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:17,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:17,023 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796276829_0001_1, initTime=1452796277021
2016-01-14 18:31:17,023 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796276829_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:17,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:DAG_STARTED]: dagID=dag_1452796276829_0001_1, startTime=1452796277024
2016-01-14 18:31:17,024 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:17,024 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796276829_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:17,024 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B249B2C1A2FF44A680C8F9E2C4AA33E0 : {6B8C8784C35E42AC877BB612BB4E4A85={InputName=6B8C8784C35E42AC877BB612BB4E4A85}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:17,024 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6B8C8784C35E42AC877BB612BB4E4A85, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:17,024 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0]
2016-01-14 18:31:17,024 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0]
2016-01-14 18:31:17,024 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0]
2016-01-14 18:31:17,025 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0]. Starting root input initializers: 1
2016-01-14 18:31:17,025 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:17,025 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:17,027 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03]
2016-01-14 18:31:17,027 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6B8C8784C35E42AC877BB612BB4E4A85 on vertex vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0]
2016-01-14 18:31:17,029 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1452796276829_0001, dagName=[346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770] every/(1/1) ...lyplatform/-1_'field'_ALL
2016-01-14 18:31:17,032 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:17,032 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6B8C8784C35E42AC877BB612BB4E4A85 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:17,034 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:17,035 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:17,035 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03]
2016-01-14 18:31:17,035 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03]
2016-01-14 18:31:17,035 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03]
2016-01-14 18:31:17,035 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=E4B1F73DF2104E83BE5271F42F548F5E, vertexId=vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:17,037 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=E4B1F73DF2104E83BE5271F42F548F5E, vertexId=vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03]
2016-01-14 18:31:17,043 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=E4B1F73DF2104E83BE5271F42F548F5E, vertexName=BC8C1C9C2B8B493E8129604F932CFB03, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:17,043 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=E4B1F73DF2104E83BE5271F42F548F5E, vertexId=vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03]
2016-01-14 18:31:17,037 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:17,048 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:17,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BC8C1C9C2B8B493E8129604F932CFB03, vertexId=vertex_1452796276829_0001_1_01, initRequestedTime=1452796277027, initedTime=1452796277048, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:17,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:17,049 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6B8C8784C35E42AC877BB612BB4E4A85 on vertex vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0]
2016-01-14 18:31:17,049 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:17,051 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:17,051 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6B8C8784C35E42AC877BB612BB4E4A85=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:17,052 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0] parallelism set to 1
2016-01-14 18:31:17,053 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:17,054 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796276829_0001_1_00, eventCount=1
2016-01-14 18:31:17,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:17,056 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B249B2C1A2FF44A680C8F9E2C4AA33E0, vertexId=vertex_1452796276829_0001_1_00, initRequestedTime=1452796277024, initedTime=1452796277054, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:17,057 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0] with distanceFromRoot: 0
2016-01-14 18:31:17,057 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:17,057 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B249B2C1A2FF44A680C8F9E2C4AA33E0
2016-01-14 18:31:17,057 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0]
2016-01-14 18:31:17,057 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796276829_0001, with dag id: dag_1452796276829_0001_1
2016-01-14 18:31:17,058 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:17,058 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796276829_0001_1_00, startRequestedTime=1452796277057, startedTime=1452796277057
2016-01-14 18:31:17,058 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:17,059 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:17,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:TASK_STARTED]: vertexName=B249B2C1A2FF44A680C8F9E2C4AA33E0, taskId=task_1452796276829_0001_1_00_000000, scheduledTime=1452796277059, launchTime=1452796277059
2016-01-14 18:31:17,059 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796276829_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:17,059 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796276829_0001_1_00 for vertex: vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03] numStartedSources: 1 numSources: 1
2016-01-14 18:31:17,060 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03] with distanceFromRoot: 1
2016-01-14 18:31:17,063 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0]
2016-01-14 18:31:17,063 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B249B2C1A2FF44A680C8F9E2C4AA33E0 in vertex: BC8C1C9C2B8B493E8129604F932CFB03
2016-01-14 18:31:17,063 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BC8C1C9C2B8B493E8129604F932CFB03 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:17,063 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:17,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796276829_0001_1_01, startRequestedTime=1452796277063, startedTime=1452796277063
2016-01-14 18:31:17,064 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:17,065 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: B249B2C1A2FF44A680C8F9E2C4AA33E0, VertexParallelism: 1, TaskAttemptID:attempt_1452796276829_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6B8C8784C35E42AC877BB612BB4E4A85, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BC8C1C9C2B8B493E8129604F932CFB03, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:17,065 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796276829_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:17,067 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:17,069 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:17,070 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:17,071 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:17,071 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796276829_0001_00_000001
2016-01-14 18:31:17,071 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:17,074 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796276829_0001_00_000001
2016-01-14 18:31:17,074 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:17,076 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:17,076 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796276829_0001_00_000001, launchTime=1452796277075
2016-01-14 18:31:17,077 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796276829_0001_00_000001
2016-01-14 18:31:17,078 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:17,088 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796276829_0001_1_00_000000_0] to container: [container_1452796276829_0001_00_000001]
2016-01-14 18:31:17,088 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:17,088 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796276829_0001_00_000001 given task: attempt_1452796276829_0001_1_00_000000_0
2016-01-14 18:31:17,089 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796276829_0001_1_00_000000_0] started. Is using containerId: [container_1452796276829_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:17,089 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796276829_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796276829_0001_1_00_000000_0
2016-01-14 18:31:17,090 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:17 Starting to run new task attempt: attempt_1452796276829_0001_1_00_000000_0
2016-01-14 18:31:17,090 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796276829_0001_1_00_000000_0
2016-01-14 18:31:17,090 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:17,090 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B249B2C1A2FF44A680C8F9E2C4AA33E0, taskAttemptId=attempt_1452796276829_0001_1_00_000000_0, startTime=1452796277089, containerId=container_1452796276829_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796276829_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:17,090 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:17,090 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:17,091 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:17,091 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:17,091 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:17,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: B249B2C1A2FF44A680C8F9E2C4AA33E0, VertexParallelism: 1, TaskAttemptID:attempt_1452796276829_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6B8C8784C35E42AC877BB612BB4E4A85, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BC8C1C9C2B8B493E8129604F932CFB03, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:17,091 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:17,092 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796276829_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:17,092 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796276829_0001_1_00_000000_0
2016-01-14 18:31:17,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:17,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:17,093 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796276829_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:17,098 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796276829_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:17,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:17,098 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6B8C8784C35E42AC877BB612BB4E4A85, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:17,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:17,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BC8C1C9C2B8B493E8129604F932CFB03, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:17,099 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6B8C8784C35E42AC877BB612BB4E4A85
2016-01-14 18:31:17,100 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6B8C8784C35E42AC877BB612BB4E4A85
2016-01-14 18:31:17,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:17,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BC8C1C9C2B8B493E8129604F932CFB03
2016-01-14 18:31:17,102 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:17,102 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6B8C8784C35E42AC877BB612BB4E4A85
2016-01-14 18:31:17,102 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6B8C8784C35E42AC877BB612BB4E4A85
2016-01-14 18:31:17,103 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:17,104 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:17,104 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BC8C1C9C2B8B493E8129604F932CFB03
2016-01-14 18:31:17,106 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BC8C1C9C2B8B493E8129604F932CFB03
2016-01-14 18:31:17,115 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B249B2C1A2FF44A680C8F9E2C4AA33E0, ordinal: 0
2016-01-14 18:31:17,115 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B249B2C1A2FF44A680C8F9E2C4AA33E0, mem on start (mb), free: 140, total: 485, max: 672
2016-01-14 18:31:17,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:17,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:17,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:17,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:17,117 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:17,118 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:17,119 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:17,120 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:17,120 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:17,120 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:17,120 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6B8C8784C35E42AC877BB612BB4E4A85, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:17,120 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BC8C1C9C2B8B493E8129604F932CFB03, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:17,120 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:17,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6B8C8784C35E42AC877BB612BB4E4A85
2016-01-14 18:31:17,122 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:17,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6B8C8784C35E42AC877BB612BB4E4A85
2016-01-14 18:31:17,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6B8C8784C35E42AC877BB612BB4E4A85 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:17,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:17,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:17,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:17,124 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796276829_0001_1_00_000000_0
2016-01-14 18:31:17,124 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:17,126 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4f7395ee
2016-01-14 18:31:17,128 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4f7395ee
2016-01-14 18:31:17,128 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:17,128 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:17,128 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:17,130 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 6B8C8784C35E42AC877BB612BB4E4A85
2016-01-14 18:31:17,130 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: BA38161523894CD8BFE7C43798DE4581
2016-01-14 18:31:17,130 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] BA38161523894CD8BFE7C43798DE4581
2016-01-14 18:31:17,131 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2cd714bd; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5c929b4a; comparator=cascading.tuple.hadoop.util.TupleComparator@1d1146c1; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:17,132 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:17,132 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:17,133 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:17,161 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:17,161 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:17,161 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:17,162 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:17,162 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:17,163 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:17,163 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:17,163 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B249B2C1A2FF44A680C8F9E2C4AA33E0, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:17,166 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796276829_0001_1_00_000000_0
2016-01-14 18:31:17,166 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B249B2C1A2FF44A680C8F9E2C4AA33E0, mem on close (mb), free: 162, total: 490, max: 672
2016-01-14 18:31:17,166 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:17,166 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:17,166 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:17,167 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:17,168 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:17 Completed running task attempt: attempt_1452796276829_0001_1_00_000000_0
2016-01-14 18:31:17,172 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796276829_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:17,173 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796276829_0001_1_00, eventCount=1
2016-01-14 18:31:17,173 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796277163, Process_Duration=3, Process_End_Time=1452796277166, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:17,173 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:17,173 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:17,174 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796276829_0001_00_000001
2016-01-14 18:31:17,174 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B249B2C1A2FF44A680C8F9E2C4AA33E0, taskAttemptId=attempt_1452796276829_0001_1_00_000000_0, startTime=1452796277089, finishTime=1452796277173, timeTaken=84, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796277163, Process_Duration=3, Process_End_Time=1452796277166, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:17,174 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796276829_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:17,174 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:17,175 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796276829_0001_1_00_000000_0
2016-01-14 18:31:17,175 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:17,176 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:TASK_FINISHED]: vertexName=B249B2C1A2FF44A680C8F9E2C4AA33E0, taskId=task_1452796276829_0001_1_00_000000, startTime=1452796277089, finishTime=1452796277175, timeTaken=86, status=SUCCEEDED, successfulAttemptID=attempt_1452796276829_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796277163, Process_Duration=3, Process_End_Time=1452796277166, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:17,176 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796276829_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:17,176 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0] : 1
2016-01-14 18:31:17,176 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0]
2016-01-14 18:31:17,176 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:17,178 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:17,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:VERTEX_FINISHED]: vertexName=B249B2C1A2FF44A680C8F9E2C4AA33E0, vertexId=vertex_1452796276829_0001_1_00, initRequestedTime=1452796277024, initedTime=1452796277054, startRequestedTime=1452796277057, startedTime=1452796277057, finishTime=1452796277176, timeTaken=119, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796277163, Process_Duration=3, Process_End_Time=1452796277166, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796277089, firstTasksToStart=[ task_1452796276829_0001_1_00_000000 ], lastTaskFinishTime=1452796277173, lastTasksToFinish=[ task_1452796276829_0001_1_00_000000 ], minTaskDuration=84, maxTaskDuration=84, avgTaskDuration=84.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796276829_0001_1_00_000000 ], longestDurationTasks=[ task_1452796276829_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:17,179 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:17,179 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03] attempt: attempt_1452796276829_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:17,179 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BC8C1C9C2B8B493E8129604F932CFB03
2016-01-14 18:31:17,181 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03]
2016-01-14 18:31:17,182 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:17,182 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796276829_0001_00_000001, stoppedTime=1452796277182, exitStatus=0
2016-01-14 18:31:17,182 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:17,182 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:17,182 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:17,184 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796276829_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:17,185 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796276829_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:17,185 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:TASK_STARTED]: vertexName=BC8C1C9C2B8B493E8129604F932CFB03, taskId=task_1452796276829_0001_1_01_000000, scheduledTime=1452796277184, launchTime=1452796277184
2016-01-14 18:31:17,185 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796276829_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:17,186 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796276829_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:17,185 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:17,186 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03]
2016-01-14 18:31:17,185 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796276829_0001_00_000001, Exiting
2016-01-14 18:31:17,186 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796276829_0001_00_000001
2016-01-14 18:31:17,186 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796276829_0001_00_000001
2016-01-14 18:31:17,188 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:17,188 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: BC8C1C9C2B8B493E8129604F932CFB03, VertexParallelism: 1, TaskAttemptID:attempt_1452796276829_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B249B2C1A2FF44A680C8F9E2C4AA33E0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E4B1F73DF2104E83BE5271F42F548F5E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:17,189 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796276829_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:17,189 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:17,188 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796276829_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:17,190 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796276829_0001_00_000001 completed successfully
2016-01-14 18:31:17,190 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796276829_0001_00_000002
2016-01-14 18:31:17,190 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:17,191 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:17,192 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796276829_0001_00_000002
2016-01-14 18:31:17,193 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:17,198 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:17,199 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:17,200 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796276829_0001_00_000002, launchTime=1452796277198
2016-01-14 18:31:17,201 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796276829_0001_00_000002
2016-01-14 18:31:17,202 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796276829_0001_1_01_000000_0] to container: [container_1452796276829_0001_00_000002]
2016-01-14 18:31:17,202 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:17,203 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796276829_0001_1_01_000000_0] started. Is using containerId: [container_1452796276829_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:17,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:17,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BC8C1C9C2B8B493E8129604F932CFB03, taskAttemptId=attempt_1452796276829_0001_1_01_000000_0, startTime=1452796277202, containerId=container_1452796276829_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796276829_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:17,203 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796276829_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:17,204 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796276829_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:17,204 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796276829_0001_00_000002 given task: attempt_1452796276829_0001_1_01_000000_0
2016-01-14 18:31:17,204 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796276829_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796276829_0001_1_01_000000_0
2016-01-14 18:31:17 Starting to run new task attempt: attempt_1452796276829_0001_1_01_000000_0
2016-01-14 18:31:17,205 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796276829_0001_1_01_000000_0
2016-01-14 18:31:17,205 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:17,208 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:17,209 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:17,210 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:17,211 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:17,211 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:17,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770] every/(1/1) ...lyplatform/-1_'field'_ALL, VertexName: BC8C1C9C2B8B493E8129604F932CFB03, VertexParallelism: 1, TaskAttemptID:attempt_1452796276829_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B249B2C1A2FF44A680C8F9E2C4AA33E0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=E4B1F73DF2104E83BE5271F42F548F5E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:17,212 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:17,212 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796276829_0001_1_01_000000_0
2016-01-14 18:31:17,213 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:17,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:17,214 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796276829_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:17,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:17,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E4B1F73DF2104E83BE5271F42F548F5E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:17,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:17,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E4B1F73DF2104E83BE5271F42F548F5E
2016-01-14 18:31:17,216 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:17,216 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: E4B1F73DF2104E83BE5271F42F548F5E
2016-01-14 18:31:17,218 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:17,220 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B249B2C1A2FF44A680C8F9E2C4AA33E0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:17,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:17,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B249B2C1A2FF44A680C8F9E2C4AA33E0
2016-01-14 18:31:17,228 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:17,228 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:17,229 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B249B2C1A2FF44A680C8F9E2C4AA33E0
2016-01-14 18:31:17,231 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BC8C1C9C2B8B493E8129604F932CFB03, ordinal: 1
2016-01-14 18:31:17,231 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC8C1C9C2B8B493E8129604F932CFB03, mem on start (mb), free: 155, total: 490, max: 672
2016-01-14 18:31:17,231 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:17,231 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:17,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B249B2C1A2FF44A680C8F9E2C4AA33E0
2016-01-14 18:31:17,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:17,233 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:17,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E4B1F73DF2104E83BE5271F42F548F5E
2016-01-14 18:31:17,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:17,233 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:17,233 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:17,234 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:17,234 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:17,234 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:17,234 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:17,234 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E4B1F73DF2104E83BE5271F42F548F5E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:17,235 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B249B2C1A2FF44A680C8F9E2C4AA33E0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:17,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:17,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B249B2C1A2FF44A680C8F9E2C4AA33E0 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:17,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:17,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:17,237 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B249B2C1A2FF44A680C8F9E2C4AA33E0
2016-01-14 18:31:17,239 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:17,239 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:17,240 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B249B2C1A2FF44A680C8F9E2C4AA33E0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:17,240 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:17,240 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:17,240 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:17,241 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B249B2C1A2FF44A680C8F9E2C4AA33E0
2016-01-14 18:31:17,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:17,242 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796276829_0001_1_01_000000_0
2016-01-14 18:31:17,242 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796276829_0001_1_00_000000_0_10021, runDuration: 70070, ]
2016-01-14 18:31:17,245 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796276829_0001_1_00_000000_0_10021] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796277245, TimeTaken=2, Rate=0.14 MB/s
2016-01-14 18:31:17,245 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B249B2C1A2FF44A680C8F9E2C4AA33E0
2016-01-14 18:31:17,246 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:17,246 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:17,246 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B249B2C1A2FF44A680C8F9E2C4AA33E0] #4 in 3ms
2016-01-14 18:31:17,247 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B249B2C1A2FF44A680C8F9E2C4AA33E0] #4, status:false, isInterrupted:false
2016-01-14 18:31:17,250 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:17,250 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:17,250 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:17,250 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:17,251 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:17,251 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B249B2C1A2FF44A680C8F9E2C4AA33E0
2016-01-14 18:31:17,252 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:17,252 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: BA38161523894CD8BFE7C43798DE4581
2016-01-14 18:31:17,252 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL"], id: E4B1F73DF2104E83BE5271F42F548F5E
2016-01-14 18:31:17,253 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_ALL"]
2016-01-14 18:31:17,253 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] BA38161523894CD8BFE7C43798DE4581, for 1 inputs
2016-01-14 18:31:17,253 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BC8C1C9C2B8B493E8129604F932CFB03, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:17,254 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3e87adf; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@13dac87a
2016-01-14 18:31:17,258 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:17,259 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:17,260 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796276829_0001_1_01_000000_0
2016-01-14 18:31:17,260 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796276829_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:17,261 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796276829_0001_1_01_000000_0
2016-01-14 18:31:17,261 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BC8C1C9C2B8B493E8129604F932CFB03, mem on close (mb), free: 152, total: 490, max: 672
2016-01-14 18:31:17,261 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B249B2C1A2FF44A680C8F9E2C4AA33E0
2016-01-14 18:31:17,264 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:17,264 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:17,264 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:17,265 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:17,265 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:17 Completed running task attempt: attempt_1452796276829_0001_1_01_000000_0
2016-01-14 18:31:17,266 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796276829_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:17,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:17,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BC8C1C9C2B8B493E8129604F932CFB03, taskAttemptId=attempt_1452796276829_0001_1_01_000000_0, startTime=1452796277202, finishTime=1452796277267, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796277253, Process_Duration=8, Process_End_Time=1452796277261, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:17,267 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796276829_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:17,268 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796276829_0001_1_01_000000_0
2016-01-14 18:31:17,268 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:17,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:TASK_FINISHED]: vertexName=BC8C1C9C2B8B493E8129604F932CFB03, taskId=task_1452796276829_0001_1_01_000000, startTime=1452796277202, finishTime=1452796277268, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1452796276829_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796277253, Process_Duration=8, Process_End_Time=1452796277261, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:17,269 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796276829_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:17,269 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03] : 1
2016-01-14 18:31:17,269 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03]
2016-01-14 18:31:17,269 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:17,271 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:17,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:VERTEX_FINISHED]: vertexName=BC8C1C9C2B8B493E8129604F932CFB03, vertexId=vertex_1452796276829_0001_1_01, initRequestedTime=1452796277027, initedTime=1452796277048, startRequestedTime=1452796277063, startedTime=1452796277063, finishTime=1452796277269, timeTaken=206, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796277253, Process_Duration=8, Process_End_Time=1452796277261, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796277202, firstTasksToStart=[ task_1452796276829_0001_1_01_000000 ], lastTaskFinishTime=1452796277267, lastTasksToFinish=[ task_1452796276829_0001_1_01_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796276829_0001_1_01_000000 ], longestDurationTasks=[ task_1452796276829_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:17,271 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:17,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796277253, Process_Duration=8, Process_End_Time=1452796277261, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:17,272 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:17,273 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:17,273 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:17,274 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796276829_0001_00_000002, stoppedTime=1452796277273, exitStatus=0
2016-01-14 18:31:17,274 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:17,274 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:17,274 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796276829_0001_1
2016-01-14 18:31:17,274 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:17,275 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796276829_0001_1
2016-01-14 18:31:17,275 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796276829_0001_1_00 [B249B2C1A2FF44A680C8F9E2C4AA33E0]
2016-01-14 18:31:17,274 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796276829_0001_00_000002
2016-01-14 18:31:17,275 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: E4B1F73DF2104E83BE5271F42F548F5E for vertex: vertex_1452796276829_0001_1_01 [BC8C1C9C2B8B493E8129604F932CFB03]
2016-01-14 18:31:17,275 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796276829_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:17,276 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796276829_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:17,276 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796276829_0001_00_000002, Exiting
2016-01-14 18:31:17,276 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796276829_0001_00_000002
2016-01-14 18:31:17,276 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796276829_0001_00_000002
2016-01-14 18:31:17,279 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796276829_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:17,279 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796276829_0001_00_000002 completed successfully
2016-01-14 18:31:17,287 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796276829_0001_1, queueSize=0
2016-01-14 18:31:17,287 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:17,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796276829_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796276829_0001_1, startTime=1452796277024, finishTime=1452796277284, timeTaken=260, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=571, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=14, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1028653056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592554416, Process_Duration=11, Process_End_Time=2905592554427, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:17,289 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796276829_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:17,289 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796276829_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:17,290 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796276829_0001_00_000002
2016-01-14 18:31:17,290 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796276829_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:17 Completed Dag: dag_1452796276829_0001_1
2016-01-14 18:31:17,292 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796276829_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:17,293 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796276829_0001_1_post
2016-01-14 18:31:17,293 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:17,293 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:17,298 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_'field'_ALL, applicationId=application_1452796276829_0001
2016-01-14 18:31:17,300 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:17,300 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:17,301 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:17,301 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:17,301 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:17,302 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:17,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:17,304 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:17,304 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:17,304 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:17,305 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:17,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:17,306 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:17,312 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/346F2C24DBE045018960D62EDAC61BBC/D92C1E878AE74BBC817C23582D1DB770/.tez/application_1452796276829_0001
2016-01-14 18:31:17,313 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.513
2016-01-14 18:31:17,397 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_ALL
2016-01-14 18:31:17,424 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:17,424 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:17,461 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-01-14 18:31:17,465 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.011
2016-01-14 18:31:17,466 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:17,466 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:17,467 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_REPLACE
2016-01-14 18:31:17,503 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:17,503 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:17,548 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 18:31:17,567 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.035
2016-01-14 18:31:17,568 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:17,568 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:17,568 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:17,587 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:17,596 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP
2016-01-14 18:31:17,654 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:17,654 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:17,654 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP"]
2016-01-14 18:31:17,654 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:17,654 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:17,654 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:17,655 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/-1_'field'_SWAP
2016-01-14 18:31:17,655 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:17,670 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:17,675 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:17,675 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:17,675 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:17,685 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547/.tez/application_1452796277675_0001 doesn't exist and is created
2016-01-14 18:31:17,699 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:17,699 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:17,700 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547/.tez/application_1452796277675_0001
2016-01-14 18:31:17,703 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796277675_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:17,719 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:17,720 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:17,722 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:17,722 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:17,722 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547/.tez/application_1452796277675_0001/localmode-log-dir
2016-01-14 18:31:17,723 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547/.tez/application_1452796277675_0001/localmode-log-dir/history.txt.appattempt_1452796277675_0001_000000, maxErrors=10
2016-01-14 18:31:17,723 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:17,723 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796277675_0001_000000, appSubmitTime=1452796277703, launchTime=1452796277703
2016-01-14 18:31:17,731 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:17,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:17,733 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:17,733 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:17,737 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:53959
2016-01-14 18:31:17,738 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:17,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796277675_0001_000000, startTime=1452796277740
2016-01-14 18:31:17,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:17,740 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:17,800 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:17,800 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:17,800 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1452796277675_0001, dagName=[8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-01-14 18:31:17,838 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-01-14 18:31:17,840 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796277675_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547/.tez/application_1452796277675_0001/localmode-log-dir/dag_1452796277675_0001_1.dot
2016-01-14 18:31:17,846 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547/.tez/application_1452796277675_0001/localmode-log-dir/dag_1452796277675_0001_1-tez-dag.pb.txt
2016-01-14 18:31:17,877 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796277675_0001_1
2016-01-14 18:31:17,878 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:17,878 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-01-14 18:31:17 Running Dag: dag_1452796277675_0001_1
2016-01-14 18:31:17,892 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796277675_0001_1, submitTime=1452796277838
2016-01-14 18:31:17,892 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:17,893 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F97B2B6754D8439C9FCCB088B38C5A53
2016-01-14 18:31:17,893 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:17,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:17,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796277675_0001_1, initTime=1452796277892
2016-01-14 18:31:17,895 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796277675_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:17,895 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:DAG_STARTED]: dagID=dag_1452796277675_0001_1, startTime=1452796277895
2016-01-14 18:31:17,896 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:17,896 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796277675_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:17,896 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D4A545E52D354EA7B3D8AA0980894659 : {51EA20402FB64E828465E64A92EBB9F6={InputName=51EA20402FB64E828465E64A92EBB9F6}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:17,896 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 51EA20402FB64E828465E64A92EBB9F6, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:17,896 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659]
2016-01-14 18:31:17,896 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659]
2016-01-14 18:31:17,896 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659]
2016-01-14 18:31:17,897 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659]. Starting root input initializers: 1
2016-01-14 18:31:17,897 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:17,898 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:17,899 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53]
2016-01-14 18:31:17,902 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 51EA20402FB64E828465E64A92EBB9F6 on vertex vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659]
2016-01-14 18:31:17,904 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1452796277675_0001, dagName=[8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547] every/(1/1) ...yplatform/-1_'field'_SWAP
2016-01-14 18:31:17,906 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:17,906 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53]
2016-01-14 18:31:17,906 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53]
2016-01-14 18:31:17,906 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53]
2016-01-14 18:31:17,906 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5321EA7F4A3C4BF7BCE7B29D63A3920D, vertexId=vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:17,906 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5321EA7F4A3C4BF7BCE7B29D63A3920D, vertexId=vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53]
2016-01-14 18:31:17,910 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:17,913 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 51EA20402FB64E828465E64A92EBB9F6 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:17,914 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:17,915 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:17,913 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5321EA7F4A3C4BF7BCE7B29D63A3920D, vertexName=F97B2B6754D8439C9FCCB088B38C5A53, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:17,916 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5321EA7F4A3C4BF7BCE7B29D63A3920D, vertexId=vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53]
2016-01-14 18:31:17,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F97B2B6754D8439C9FCCB088B38C5A53, vertexId=vertex_1452796277675_0001_1_01, initRequestedTime=1452796277899, initedTime=1452796277917, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:17,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:17,917 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:17,919 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:17,920 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 51EA20402FB64E828465E64A92EBB9F6 on vertex vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659]
2016-01-14 18:31:17,921 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:17,921 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {51EA20402FB64E828465E64A92EBB9F6=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:17,921 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659] parallelism set to 1
2016-01-14 18:31:17,921 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:17,922 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796277675_0001_1_00, eventCount=1
2016-01-14 18:31:17,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D4A545E52D354EA7B3D8AA0980894659, vertexId=vertex_1452796277675_0001_1_00, initRequestedTime=1452796277896, initedTime=1452796277922, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:17,924 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659] with distanceFromRoot: 0
2016-01-14 18:31:17,924 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:17,926 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:17,927 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D4A545E52D354EA7B3D8AA0980894659
2016-01-14 18:31:17,927 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659]
2016-01-14 18:31:17,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796277675_0001_1_00, startRequestedTime=1452796277927, startedTime=1452796277927
2016-01-14 18:31:17,929 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:17,929 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:17,930 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:17,930 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:TASK_STARTED]: vertexName=D4A545E52D354EA7B3D8AA0980894659, taskId=task_1452796277675_0001_1_00_000000, scheduledTime=1452796277930, launchTime=1452796277930
2016-01-14 18:31:17,930 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796277675_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:17,931 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796277675_0001_1_00 for vertex: vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53] numStartedSources: 1 numSources: 1
2016-01-14 18:31:17,931 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53] with distanceFromRoot: 1
2016-01-14 18:31:17,932 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659]
2016-01-14 18:31:17,932 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D4A545E52D354EA7B3D8AA0980894659 in vertex: F97B2B6754D8439C9FCCB088B38C5A53
2016-01-14 18:31:17,932 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F97B2B6754D8439C9FCCB088B38C5A53 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:17,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:17,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796277675_0001_1_01, startRequestedTime=1452796277932, startedTime=1452796277932
2016-01-14 18:31:17,934 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:17,934 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: D4A545E52D354EA7B3D8AA0980894659, VertexParallelism: 1, TaskAttemptID:attempt_1452796277675_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=51EA20402FB64E828465E64A92EBB9F6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F97B2B6754D8439C9FCCB088B38C5A53, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:17,935 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796277675_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:17,935 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:17,936 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:17,937 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:17,937 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:17,937 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796277675_0001_00_000001
2016-01-14 18:31:17,938 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:17,939 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796277675_0001_00_000001
2016-01-14 18:31:17,940 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:17,940 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:17,941 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:17,941 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796277675_0001_00_000001, launchTime=1452796277940
2016-01-14 18:31:17,941 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796277675_0001_00_000001
2016-01-14 18:31:17,942 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796277675_0001_1_00_000000_0] to container: [container_1452796277675_0001_00_000001]
2016-01-14 18:31:17,942 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:17,942 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796277675_0001_00_000001 given task: attempt_1452796277675_0001_1_00_000000_0
2016-01-14 18:31:17,943 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796277675_0001, with dag id: dag_1452796277675_0001_1
2016-01-14 18:31:17,943 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796277675_0001_1_00_000000_0] started. Is using containerId: [container_1452796277675_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:17,943 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D4A545E52D354EA7B3D8AA0980894659, taskAttemptId=attempt_1452796277675_0001_1_00_000000_0, startTime=1452796277943, containerId=container_1452796277675_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796277675_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:17,943 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796277675_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:17,944 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796277675_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:17,944 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:17,943 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796277675_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796277675_0001_1_00_000000_0
2016-01-14 18:31:17 Starting to run new task attempt: attempt_1452796277675_0001_1_00_000000_0
2016-01-14 18:31:17,949 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796277675_0001_1_00_000000_0
2016-01-14 18:31:17,949 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:17,949 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:17,949 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:17,950 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:17,950 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:17,950 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:17,951 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: D4A545E52D354EA7B3D8AA0980894659, VertexParallelism: 1, TaskAttemptID:attempt_1452796277675_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=51EA20402FB64E828465E64A92EBB9F6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F97B2B6754D8439C9FCCB088B38C5A53, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:17,951 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:17,953 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796277675_0001_1_00_000000_0
2016-01-14 18:31:17,953 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:17,953 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:17,954 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:17,955 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=51EA20402FB64E828465E64A92EBB9F6, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:17,955 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796277675_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:17,956 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:17,957 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 51EA20402FB64E828465E64A92EBB9F6
2016-01-14 18:31:17,957 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 51EA20402FB64E828465E64A92EBB9F6
2016-01-14 18:31:17,958 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:17,959 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F97B2B6754D8439C9FCCB088B38C5A53, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:17,960 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:17,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F97B2B6754D8439C9FCCB088B38C5A53
2016-01-14 18:31:17,962 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:17,962 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 51EA20402FB64E828465E64A92EBB9F6
2016-01-14 18:31:17,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 51EA20402FB64E828465E64A92EBB9F6
2016-01-14 18:31:17,964 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:17,964 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F97B2B6754D8439C9FCCB088B38C5A53
2016-01-14 18:31:17,965 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F97B2B6754D8439C9FCCB088B38C5A53
2016-01-14 18:31:17,966 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D4A545E52D354EA7B3D8AA0980894659, ordinal: 0
2016-01-14 18:31:17,966 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D4A545E52D354EA7B3D8AA0980894659, mem on start (mb), free: 242, total: 490, max: 672
2016-01-14 18:31:17,966 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:17,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:17,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:17,967 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:17,967 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:17,967 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:17,967 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:17,968 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:17,968 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:17,968 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:17,968 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 51EA20402FB64E828465E64A92EBB9F6, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:17,968 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F97B2B6754D8439C9FCCB088B38C5A53, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:17,968 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:17,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 51EA20402FB64E828465E64A92EBB9F6 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:17,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:17,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:17,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 51EA20402FB64E828465E64A92EBB9F6
2016-01-14 18:31:17,969 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:17,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 51EA20402FB64E828465E64A92EBB9F6
2016-01-14 18:31:17,969 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:17,970 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796277675_0001_1_00_000000_0
2016-01-14 18:31:17,971 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:17,973 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:17,974 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7258adc3
2016-01-14 18:31:17,974 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 51EA20402FB64E828465E64A92EBB9F6
2016-01-14 18:31:17,974 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C08817D4DF9A4B8CA1B0E1940496F70B
2016-01-14 18:31:17,975 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7258adc3
2016-01-14 18:31:17,975 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:17,975 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:17,975 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C08817D4DF9A4B8CA1B0E1940496F70B
2016-01-14 18:31:17,976 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@280290fe; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@407d2837; comparator=cascading.tuple.hadoop.util.TupleComparator@58db53f4; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:17,977 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:17,977 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:17,977 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:17,990 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:17,991 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:17,991 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:17,991 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:17,991 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:17,992 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:17,993 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:17,993 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D4A545E52D354EA7B3D8AA0980894659, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:17,996 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796277675_0001_1_00_000000_0
2016-01-14 18:31:17,996 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D4A545E52D354EA7B3D8AA0980894659, mem on close (mb), free: 140, total: 490, max: 672
2016-01-14 18:31:17,997 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:17,997 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:17,997 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:17,997 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:18,000 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:18 Completed running task attempt: attempt_1452796277675_0001_1_00_000000_0
2016-01-14 18:31:18,003 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796277675_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:18,004 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796277675_0001_1_00, eventCount=1
2016-01-14 18:31:18,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796277994, Process_Duration=2, Process_End_Time=1452796277996, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:18,005 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D4A545E52D354EA7B3D8AA0980894659, taskAttemptId=attempt_1452796277675_0001_1_00_000000_0, startTime=1452796277943, finishTime=1452796278005, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796277994, Process_Duration=2, Process_End_Time=1452796277996, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:18,005 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:18,005 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796277675_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:18,006 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796277675_0001_1_00_000000_0
2016-01-14 18:31:18,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:18,006 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:18,007 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:18,007 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796277675_0001_00_000001
2016-01-14 18:31:18,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:TASK_FINISHED]: vertexName=D4A545E52D354EA7B3D8AA0980894659, taskId=task_1452796277675_0001_1_00_000000, startTime=1452796277943, finishTime=1452796278006, timeTaken=63, status=SUCCEEDED, successfulAttemptID=attempt_1452796277675_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796277994, Process_Duration=2, Process_End_Time=1452796277996, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:18,007 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796277675_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:18,007 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659] : 1
2016-01-14 18:31:18,008 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659]
2016-01-14 18:31:18,007 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:18,009 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:18,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:VERTEX_FINISHED]: vertexName=D4A545E52D354EA7B3D8AA0980894659, vertexId=vertex_1452796277675_0001_1_00, initRequestedTime=1452796277896, initedTime=1452796277922, startRequestedTime=1452796277927, startedTime=1452796277927, finishTime=1452796278008, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796277994, Process_Duration=2, Process_End_Time=1452796277996, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796277943, firstTasksToStart=[ task_1452796277675_0001_1_00_000000 ], lastTaskFinishTime=1452796278005, lastTasksToFinish=[ task_1452796277675_0001_1_00_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796277675_0001_1_00_000000 ], longestDurationTasks=[ task_1452796277675_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:18,009 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:18,010 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53] attempt: attempt_1452796277675_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:18,010 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F97B2B6754D8439C9FCCB088B38C5A53
2016-01-14 18:31:18,010 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53]
2016-01-14 18:31:18,011 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:18,011 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796277675_0001_00_000001, stoppedTime=1452796278011, exitStatus=0
2016-01-14 18:31:18,011 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:18,012 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:18,012 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:18,013 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:18,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:TASK_STARTED]: vertexName=F97B2B6754D8439C9FCCB088B38C5A53, taskId=task_1452796277675_0001_1_01_000000, scheduledTime=1452796278013, launchTime=1452796278013
2016-01-14 18:31:18,013 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796277675_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:18,014 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796277675_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:18,014 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53]
2016-01-14 18:31:18,014 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:18,015 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: F97B2B6754D8439C9FCCB088B38C5A53, VertexParallelism: 1, TaskAttemptID:attempt_1452796277675_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D4A545E52D354EA7B3D8AA0980894659, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=5321EA7F4A3C4BF7BCE7B29D63A3920D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:18,015 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796277675_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:18,015 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:18,016 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796277675_0001_00_000002
2016-01-14 18:31:18,016 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:18,017 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796277675_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:18,018 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796277675_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:18,018 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796277675_0001_00_000001, Exiting
2016-01-14 18:31:18,018 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796277675_0001_00_000001
2016-01-14 18:31:18,018 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796277675_0001_00_000001
2016-01-14 18:31:18,019 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796277675_0001_00_000002
2016-01-14 18:31:18,021 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796277675_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:18,021 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796277675_0001_00_000001 completed successfully
2016-01-14 18:31:18,022 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:18,022 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:18,024 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:18,024 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796277675_0001_00_000002, launchTime=1452796278024
2016-01-14 18:31:18,024 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:18,025 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796277675_0001_00_000002
2016-01-14 18:31:18,025 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796277675_0001_1_01_000000_0] to container: [container_1452796277675_0001_00_000002]
2016-01-14 18:31:18,025 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:18,026 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796277675_0001_00_000002 given task: attempt_1452796277675_0001_1_01_000000_0
2016-01-14 18:31:18,026 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796277675_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796277675_0001_1_01_000000_0
2016-01-14 18:31:18,026 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796277675_0001_1_01_000000_0] started. Is using containerId: [container_1452796277675_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:18 Starting to run new task attempt: attempt_1452796277675_0001_1_01_000000_0
2016-01-14 18:31:18,026 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796277675_0001_1_01_000000_0
2016-01-14 18:31:18,027 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:18,027 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:18,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F97B2B6754D8439C9FCCB088B38C5A53, taskAttemptId=attempt_1452796277675_0001_1_01_000000_0, startTime=1452796278026, containerId=container_1452796277675_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796277675_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:18,028 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:18,027 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:18,028 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796277675_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:18,029 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796277675_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:18,029 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:18,029 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:18,029 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:18,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547] every/(1/1) ...yplatform/-1_'field'_SWAP, VertexName: F97B2B6754D8439C9FCCB088B38C5A53, VertexParallelism: 1, TaskAttemptID:attempt_1452796277675_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D4A545E52D354EA7B3D8AA0980894659, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=5321EA7F4A3C4BF7BCE7B29D63A3920D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:18,030 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:18,030 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796277675_0001_1_01_000000_0
2016-01-14 18:31:18,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:18,031 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:18,032 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796277675_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:18,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D4A545E52D354EA7B3D8AA0980894659, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:18,032 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:18,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D4A545E52D354EA7B3D8AA0980894659
2016-01-14 18:31:18,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:18,033 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5321EA7F4A3C4BF7BCE7B29D63A3920D, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:18,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:18,034 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5321EA7F4A3C4BF7BCE7B29D63A3920D
2016-01-14 18:31:18,034 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:18,034 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5321EA7F4A3C4BF7BCE7B29D63A3920D
2016-01-14 18:31:18,035 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:18,035 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:18,035 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D4A545E52D354EA7B3D8AA0980894659
2016-01-14 18:31:18,038 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:18,038 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D4A545E52D354EA7B3D8AA0980894659
2016-01-14 18:31:18,047 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:18,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5321EA7F4A3C4BF7BCE7B29D63A3920D
2016-01-14 18:31:18,047 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F97B2B6754D8439C9FCCB088B38C5A53, ordinal: 1
2016-01-14 18:31:18,047 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F97B2B6754D8439C9FCCB088B38C5A53, mem on start (mb), free: 134, total: 490, max: 672
2016-01-14 18:31:18,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:18,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:18,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:18,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:18,048 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:18,049 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:18,049 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:18,049 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:18,050 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:18,050 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:18,050 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5321EA7F4A3C4BF7BCE7B29D63A3920D, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:18,050 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D4A545E52D354EA7B3D8AA0980894659, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:18,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:18,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D4A545E52D354EA7B3D8AA0980894659 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:18,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:18,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:18,051 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D4A545E52D354EA7B3D8AA0980894659
2016-01-14 18:31:18,053 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:18,054 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:18,054 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D4A545E52D354EA7B3D8AA0980894659 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:18,055 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:18,055 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:18,055 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:18,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D4A545E52D354EA7B3D8AA0980894659
2016-01-14 18:31:18,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:18,058 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796277675_0001_1_01_000000_0
2016-01-14 18:31:18,059 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796277675_0001_1_00_000000_0_10027, runDuration: 39787, ]
2016-01-14 18:31:18,060 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796277675_0001_1_00_000000_0_10027] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796278060, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:18,060 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D4A545E52D354EA7B3D8AA0980894659
2016-01-14 18:31:18,060 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:18,060 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:18,061 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D4A545E52D354EA7B3D8AA0980894659] #5 in 2ms
2016-01-14 18:31:18,061 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D4A545E52D354EA7B3D8AA0980894659] #5, status:false, isInterrupted:false
2016-01-14 18:31:18,061 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C08817D4DF9A4B8CA1B0E1940496F70B
2016-01-14 18:31:18,062 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP"], id: 5321EA7F4A3C4BF7BCE7B29D63A3920D
2016-01-14 18:31:18,062 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_'field'_SWAP"]
2016-01-14 18:31:18,062 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:18,062 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C08817D4DF9A4B8CA1B0E1940496F70B, for 1 inputs
2016-01-14 18:31:18,063 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:18,063 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:18,063 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:18,064 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:18,065 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D4A545E52D354EA7B3D8AA0980894659
2016-01-14 18:31:18,065 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F97B2B6754D8439C9FCCB088B38C5A53, all 1 inputs ready in: 00:00:00.002
2016-01-14 18:31:18,067 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:18,067 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@149504e6; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4374a298
2016-01-14 18:31:18,071 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:18,072 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:18,072 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796277675_0001_1_01_000000_0
2016-01-14 18:31:18,073 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796277675_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:18,074 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796277675_0001_1_01_000000_0
2016-01-14 18:31:18,074 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F97B2B6754D8439C9FCCB088B38C5A53, mem on close (mb), free: 131, total: 490, max: 672
2016-01-14 18:31:18,074 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D4A545E52D354EA7B3D8AA0980894659
2016-01-14 18:31:18,074 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:18,075 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:18,075 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:18,075 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:18,075 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:18 Completed running task attempt: attempt_1452796277675_0001_1_01_000000_0
2016-01-14 18:31:18,076 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796277675_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:18,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796278065, Process_Duration=9, Process_End_Time=1452796278074, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:18,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F97B2B6754D8439C9FCCB088B38C5A53, taskAttemptId=attempt_1452796277675_0001_1_01_000000_0, startTime=1452796278026, finishTime=1452796278078, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796278065, Process_Duration=9, Process_End_Time=1452796278074, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:18,079 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796277675_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:18,080 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:18,080 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796277675_0001_1_01_000000_0
2016-01-14 18:31:18,078 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:18,081 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:TASK_FINISHED]: vertexName=F97B2B6754D8439C9FCCB088B38C5A53, taskId=task_1452796277675_0001_1_01_000000, startTime=1452796278026, finishTime=1452796278080, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452796277675_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796278065, Process_Duration=9, Process_End_Time=1452796278074, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:18,081 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796277675_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:18,081 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53] : 1
2016-01-14 18:31:18,081 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53]
2016-01-14 18:31:18,081 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:18,083 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:18,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:VERTEX_FINISHED]: vertexName=F97B2B6754D8439C9FCCB088B38C5A53, vertexId=vertex_1452796277675_0001_1_01, initRequestedTime=1452796277899, initedTime=1452796277917, startRequestedTime=1452796277932, startedTime=1452796277932, finishTime=1452796278081, timeTaken=149, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=707, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796278065, Process_Duration=9, Process_End_Time=1452796278074, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796278026, firstTasksToStart=[ task_1452796277675_0001_1_01_000000 ], lastTaskFinishTime=1452796278078, lastTasksToFinish=[ task_1452796277675_0001_1_01_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796277675_0001_1_01_000000 ], longestDurationTasks=[ task_1452796277675_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:18,083 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:18,083 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:18,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796277675_0001_00_000002, stoppedTime=1452796278083, exitStatus=0
2016-01-14 18:31:18,084 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:18,084 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:18,084 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796277675_0001_1
2016-01-14 18:31:18,084 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796277675_0001_1
2016-01-14 18:31:18,084 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5321EA7F4A3C4BF7BCE7B29D63A3920D for vertex: vertex_1452796277675_0001_1_01 [F97B2B6754D8439C9FCCB088B38C5A53]
2016-01-14 18:31:18,084 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:18,085 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:18,086 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:18,086 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796277675_0001_00_000002
2016-01-14 18:31:18,088 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796277675_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:18,089 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796277675_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:18,089 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796277675_0001_00_000002, Exiting
2016-01-14 18:31:18,089 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796277675_0001_00_000002
2016-01-14 18:31:18,089 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796277675_0001_00_000002
2016-01-14 18:31:18,090 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796277675_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:18,091 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796277675_0001_00_000002 completed successfully
2016-01-14 18:31:18,094 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796277675_0001_1_00 [D4A545E52D354EA7B3D8AA0980894659]
2016-01-14 18:31:18,098 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796277675_0001_1, queueSize=0
2016-01-14 18:31:18,098 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:18,099 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796277675_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796277675_0001_1, startTime=1452796277895, finishTime=1452796278094, timeTaken=199, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1040, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1027604480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592556059, Process_Duration=11, Process_End_Time=2905592556070, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:18,100 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796277675_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:18,100 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796277675_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:18,100 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796277675_0001_00_000002
2016-01-14 18:31:18,100 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796277675_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:18 Completed Dag: dag_1452796277675_0001_1
2016-01-14 18:31:18,101 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796277675_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:18,101 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796277675_0001_1_post
2016-01-14 18:31:18,101 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:18,101 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:18,108 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_'field'_SWAP, applicationId=application_1452796277675_0001
2016-01-14 18:31:18,111 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:18,111 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:18,111 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:18,112 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:18,112 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:18,112 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:18,113 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:18,113 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:18,113 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:18,113 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:18,113 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:18,114 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:18,114 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:18,120 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8277EA36A5224697AF5D0317C64E5D1F/F6C3278B8E8B41539E68E2959F87A547/.tez/application_1452796277675_0001
2016-01-14 18:31:18,120 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.467
2016-01-14 18:31:18,201 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_'field'_SWAP
2016-01-14 18:31:18,227 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:18,227 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:18,275 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-01-14 18:31:18,295 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.039
2016-01-14 18:31:18,296 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:18,296 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:18,296 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:18,314 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:18,321 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1
2016-01-14 18:31:18,379 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:18,379 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:18,379 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1"]
2016-01-14 18:31:18,379 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:18,379 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:18,380 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:18,382 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/-1_UNKNOWN_-1
2016-01-14 18:31:18,382 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:18,397 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:18,401 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:18,402 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:18,402 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:18,411 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386/.tez/application_1452796278402_0001 doesn't exist and is created
2016-01-14 18:31:18,426 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:18,426 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:18,427 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386/.tez/application_1452796278402_0001
2016-01-14 18:31:18,428 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796278402_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:18,443 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:18,444 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:18,446 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:18,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:18,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386/.tez/application_1452796278402_0001/localmode-log-dir
2016-01-14 18:31:18,447 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386/.tez/application_1452796278402_0001/localmode-log-dir/history.txt.appattempt_1452796278402_0001_000000, maxErrors=10
2016-01-14 18:31:18,448 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:18,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796278402_0001_000000, appSubmitTime=1452796278428, launchTime=1452796278428
2016-01-14 18:31:18,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:18,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:18,458 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:44593
2016-01-14 18:31:18,458 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:18,458 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:18,461 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:18,464 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796278402_0001_000000, startTime=1452796278463
2016-01-14 18:31:18,464 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:18,464 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:18,526 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:18,526 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:18,526 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1452796278402_0001, dagName=[57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-01-14 18:31:18,557 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-01-14 18:31:18,558 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796278402_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386/.tez/application_1452796278402_0001/localmode-log-dir/dag_1452796278402_0001_1.dot
2016-01-14 18:31:18,559 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386/.tez/application_1452796278402_0001/localmode-log-dir/dag_1452796278402_0001_1-tez-dag.pb.txt
2016-01-14 18:31:18,581 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796278402_0001_1
2016-01-14 18:31:18,581 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:18,582 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-01-14 18:31:18 Running Dag: dag_1452796278402_0001_1
2016-01-14 18:31:18,597 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796278402_0001_1, submitTime=1452796278557
2016-01-14 18:31:18,597 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:18,598 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CE2A4478E0B04F1D8EE03D1335EE9234
2016-01-14 18:31:18,598 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:18,599 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796278402_0001_1, initTime=1452796278597
2016-01-14 18:31:18,599 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796278402_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:18,600 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:18,600 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:DAG_STARTED]: dagID=dag_1452796278402_0001_1, startTime=1452796278600
2016-01-14 18:31:18,600 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:18,601 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796278402_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:18,601 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BDAD36BEBD0541D89B5B2C0C91271A43 : {2E3A3AF73E814D08A3C9BEDD1508B169={InputName=2E3A3AF73E814D08A3C9BEDD1508B169}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:18,601 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2E3A3AF73E814D08A3C9BEDD1508B169, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:18,601 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43]
2016-01-14 18:31:18,601 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43]
2016-01-14 18:31:18,601 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43]
2016-01-14 18:31:18,602 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43]. Starting root input initializers: 1
2016-01-14 18:31:18,602 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1452796278402_0001, dagName=[57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386] every/(1/1) ...blyplatform/-1_UNKNOWN_-1
2016-01-14 18:31:18,605 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:18,606 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234]
2016-01-14 18:31:18,609 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:18,609 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234]
2016-01-14 18:31:18,610 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234]
2016-01-14 18:31:18,610 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234]
2016-01-14 18:31:18,610 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2C328B3CF3AA471C88D2E28DD7126EE3, vertexId=vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:18,610 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2C328B3CF3AA471C88D2E28DD7126EE3, vertexId=vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234]
2016-01-14 18:31:18,610 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:18,616 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2C328B3CF3AA471C88D2E28DD7126EE3, vertexName=CE2A4478E0B04F1D8EE03D1335EE9234, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:18,616 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2E3A3AF73E814D08A3C9BEDD1508B169 on vertex vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43]
2016-01-14 18:31:18,617 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2C328B3CF3AA471C88D2E28DD7126EE3, vertexId=vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234]
2016-01-14 18:31:18,620 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:18,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CE2A4478E0B04F1D8EE03D1335EE9234, vertexId=vertex_1452796278402_0001_1_01, initRequestedTime=1452796278606, initedTime=1452796278619, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:18,620 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:18,621 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:18,621 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2E3A3AF73E814D08A3C9BEDD1508B169 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:18,623 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:18,623 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:18,625 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:18,628 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2E3A3AF73E814D08A3C9BEDD1508B169 on vertex vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43]
2016-01-14 18:31:18,629 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:18,629 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2E3A3AF73E814D08A3C9BEDD1508B169=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:18,629 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796278402_0001, with dag id: dag_1452796278402_0001_1
2016-01-14 18:31:18,631 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43] parallelism set to 1
2016-01-14 18:31:18,631 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:18,632 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796278402_0001_1_00, eventCount=1
2016-01-14 18:31:18,633 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:18,633 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BDAD36BEBD0541D89B5B2C0C91271A43, vertexId=vertex_1452796278402_0001_1_00, initRequestedTime=1452796278601, initedTime=1452796278632, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:18,636 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43] with distanceFromRoot: 0
2016-01-14 18:31:18,636 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:18,638 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BDAD36BEBD0541D89B5B2C0C91271A43
2016-01-14 18:31:18,638 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43]
2016-01-14 18:31:18,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:18,641 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796278402_0001_1_00, startRequestedTime=1452796278638, startedTime=1452796278638
2016-01-14 18:31:18,641 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:18,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:18,643 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:TASK_STARTED]: vertexName=BDAD36BEBD0541D89B5B2C0C91271A43, taskId=task_1452796278402_0001_1_00_000000, scheduledTime=1452796278642, launchTime=1452796278642
2016-01-14 18:31:18,643 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796278402_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:18,643 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796278402_0001_1_00 for vertex: vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234] numStartedSources: 1 numSources: 1
2016-01-14 18:31:18,643 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234] with distanceFromRoot: 1
2016-01-14 18:31:18,644 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43]
2016-01-14 18:31:18,644 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BDAD36BEBD0541D89B5B2C0C91271A43 in vertex: CE2A4478E0B04F1D8EE03D1335EE9234
2016-01-14 18:31:18,645 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CE2A4478E0B04F1D8EE03D1335EE9234 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:18,645 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:18,646 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796278402_0001_1_01, startRequestedTime=1452796278644, startedTime=1452796278644
2016-01-14 18:31:18,646 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:18,647 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: BDAD36BEBD0541D89B5B2C0C91271A43, VertexParallelism: 1, TaskAttemptID:attempt_1452796278402_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2E3A3AF73E814D08A3C9BEDD1508B169, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CE2A4478E0B04F1D8EE03D1335EE9234, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:18,647 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796278402_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:18,648 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:18,650 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:18,650 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:18,651 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:18,651 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796278402_0001_00_000001
2016-01-14 18:31:18,651 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:18,654 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796278402_0001_00_000001
2016-01-14 18:31:18,654 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:18,655 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:18,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796278402_0001_00_000001, launchTime=1452796278655
2016-01-14 18:31:18,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:18,658 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796278402_0001_00_000001
2016-01-14 18:31:18,659 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796278402_0001_1_00_000000_0] to container: [container_1452796278402_0001_00_000001]
2016-01-14 18:31:18,659 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:18,660 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796278402_0001_1_00_000000_0] started. Is using containerId: [container_1452796278402_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:18,661 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:18,661 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BDAD36BEBD0541D89B5B2C0C91271A43, taskAttemptId=attempt_1452796278402_0001_1_00_000000_0, startTime=1452796278660, containerId=container_1452796278402_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796278402_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:18,661 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796278402_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:18,661 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796278402_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:18,662 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796278402_0001_00_000001 given task: attempt_1452796278402_0001_1_00_000000_0
2016-01-14 18:31:18,663 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796278402_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796278402_0001_1_00_000000_0
2016-01-14 18:31:18 Starting to run new task attempt: attempt_1452796278402_0001_1_00_000000_0
2016-01-14 18:31:18,665 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796278402_0001_1_00_000000_0
2016-01-14 18:31:18,665 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:18,665 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:18,666 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:18,666 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:18,666 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:18,667 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:18,667 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: BDAD36BEBD0541D89B5B2C0C91271A43, VertexParallelism: 1, TaskAttemptID:attempt_1452796278402_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2E3A3AF73E814D08A3C9BEDD1508B169, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CE2A4478E0B04F1D8EE03D1335EE9234, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:18,667 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:18,668 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796278402_0001_1_00_000000_0
2016-01-14 18:31:18,668 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:18,668 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:18,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:18,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2E3A3AF73E814D08A3C9BEDD1508B169, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:18,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:18,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2E3A3AF73E814D08A3C9BEDD1508B169
2016-01-14 18:31:18,670 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2E3A3AF73E814D08A3C9BEDD1508B169
2016-01-14 18:31:18,673 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796278402_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:18,676 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:18,676 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:18,677 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2E3A3AF73E814D08A3C9BEDD1508B169
2016-01-14 18:31:18,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2E3A3AF73E814D08A3C9BEDD1508B169
2016-01-14 18:31:18,680 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CE2A4478E0B04F1D8EE03D1335EE9234, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:18,680 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:18,681 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CE2A4478E0B04F1D8EE03D1335EE9234
2016-01-14 18:31:18,684 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:18,684 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CE2A4478E0B04F1D8EE03D1335EE9234
2016-01-14 18:31:18,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CE2A4478E0B04F1D8EE03D1335EE9234
2016-01-14 18:31:18,686 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BDAD36BEBD0541D89B5B2C0C91271A43, ordinal: 0
2016-01-14 18:31:18,687 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDAD36BEBD0541D89B5B2C0C91271A43, mem on start (mb), free: 240, total: 490, max: 672
2016-01-14 18:31:18,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:18,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:18,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:18,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:18,688 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:18,689 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:18,689 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:18,689 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:18,689 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:18,690 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:18,690 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2E3A3AF73E814D08A3C9BEDD1508B169, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:18,690 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CE2A4478E0B04F1D8EE03D1335EE9234, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:18,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:18,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2E3A3AF73E814D08A3C9BEDD1508B169
2016-01-14 18:31:18,691 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:18,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2E3A3AF73E814D08A3C9BEDD1508B169
2016-01-14 18:31:18,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2E3A3AF73E814D08A3C9BEDD1508B169 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:18,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:18,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:18,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:18,693 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796278402_0001_1_00_000000_0
2016-01-14 18:31:18,696 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:18,697 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2E3A3AF73E814D08A3C9BEDD1508B169
2016-01-14 18:31:18,698 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 5DDDC385431044578A86EA3CABE1AB38
2016-01-14 18:31:18,698 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 5DDDC385431044578A86EA3CABE1AB38
2016-01-14 18:31:18,699 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@a70ed70; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7bf4596a; comparator=cascading.tuple.hadoop.util.TupleComparator@671a7db9; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:18,699 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:18,701 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@25542294
2016-01-14 18:31:18,700 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:18,702 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:18,702 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:18,703 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@25542294
2016-01-14 18:31:18,703 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:18,703 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:18,719 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:18,720 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:18,720 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:18,722 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:18,722 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:18,724 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:18,724 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:18,725 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BDAD36BEBD0541D89B5B2C0C91271A43, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:18,730 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796278402_0001_1_00_000000_0
2016-01-14 18:31:18,731 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BDAD36BEBD0541D89B5B2C0C91271A43, mem on close (mb), free: 138, total: 490, max: 672
2016-01-14 18:31:18,731 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:18,731 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:18,731 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:18,731 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:18,737 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:18 Completed running task attempt: attempt_1452796278402_0001_1_00_000000_0
2016-01-14 18:31:18,742 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796278402_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:18,743 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796278402_0001_1_00, eventCount=1
2016-01-14 18:31:18,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796278725, Process_Duration=5, Process_End_Time=1452796278730, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=3][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:18,743 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:18,744 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:18,744 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:18,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BDAD36BEBD0541D89B5B2C0C91271A43, taskAttemptId=attempt_1452796278402_0001_1_00_000000_0, startTime=1452796278660, finishTime=1452796278743, timeTaken=83, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796278725, Process_Duration=5, Process_End_Time=1452796278730, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:18,749 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796278402_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:18,749 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796278402_0001_1_00_000000_0
2016-01-14 18:31:18,750 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796278402_0001_00_000001
2016-01-14 18:31:18,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:18,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:TASK_FINISHED]: vertexName=BDAD36BEBD0541D89B5B2C0C91271A43, taskId=task_1452796278402_0001_1_00_000000, startTime=1452796278660, finishTime=1452796278749, timeTaken=89, status=SUCCEEDED, successfulAttemptID=attempt_1452796278402_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796278725, Process_Duration=5, Process_End_Time=1452796278730, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:18,750 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796278402_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:18,750 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43] : 1
2016-01-14 18:31:18,751 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43]
2016-01-14 18:31:18,751 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:18,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:18,754 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:VERTEX_FINISHED]: vertexName=BDAD36BEBD0541D89B5B2C0C91271A43, vertexId=vertex_1452796278402_0001_1_00, initRequestedTime=1452796278601, initedTime=1452796278632, startRequestedTime=1452796278638, startedTime=1452796278638, finishTime=1452796278751, timeTaken=113, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796278725, Process_Duration=5, Process_End_Time=1452796278730, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796278660, firstTasksToStart=[ task_1452796278402_0001_1_00_000000 ], lastTaskFinishTime=1452796278743, lastTasksToFinish=[ task_1452796278402_0001_1_00_000000 ], minTaskDuration=83, maxTaskDuration=83, avgTaskDuration=83.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796278402_0001_1_00_000000 ], longestDurationTasks=[ task_1452796278402_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:18,754 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:18,755 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234] attempt: attempt_1452796278402_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:18,756 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CE2A4478E0B04F1D8EE03D1335EE9234
2016-01-14 18:31:18,756 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234]
2016-01-14 18:31:18,757 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:18,758 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796278402_0001_00_000001, stoppedTime=1452796278758, exitStatus=0
2016-01-14 18:31:18,758 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:18,758 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:18,759 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:18,760 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:18,760 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:TASK_STARTED]: vertexName=CE2A4478E0B04F1D8EE03D1335EE9234, taskId=task_1452796278402_0001_1_01_000000, scheduledTime=1452796278759, launchTime=1452796278759
2016-01-14 18:31:18,760 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796278402_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:18,760 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796278402_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:18,760 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796278402_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:18,761 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234]
2016-01-14 18:31:18,761 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:18,762 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: CE2A4478E0B04F1D8EE03D1335EE9234, VertexParallelism: 1, TaskAttemptID:attempt_1452796278402_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BDAD36BEBD0541D89B5B2C0C91271A43, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2C328B3CF3AA471C88D2E28DD7126EE3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:18,762 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796278402_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:18,761 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796278402_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:18,763 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796278402_0001_00_000001, Exiting
2016-01-14 18:31:18,763 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796278402_0001_00_000001
2016-01-14 18:31:18,763 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796278402_0001_00_000001
2016-01-14 18:31:18,767 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:18,768 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796278402_0001_00_000002
2016-01-14 18:31:18,768 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:18,771 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796278402_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:18,771 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796278402_0001_00_000001 completed successfully
2016-01-14 18:31:18,772 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:18,772 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796278402_0001_00_000002
2016-01-14 18:31:18,773 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:18,777 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:18,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:18,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796278402_0001_00_000002, launchTime=1452796278777
2016-01-14 18:31:18,778 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796278402_0001_00_000002
2016-01-14 18:31:18,779 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796278402_0001_1_01_000000_0] to container: [container_1452796278402_0001_00_000002]
2016-01-14 18:31:18,779 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:18,780 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796278402_0001_1_01_000000_0] started. Is using containerId: [container_1452796278402_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:18,780 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:18,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CE2A4478E0B04F1D8EE03D1335EE9234, taskAttemptId=attempt_1452796278402_0001_1_01_000000_0, startTime=1452796278780, containerId=container_1452796278402_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796278402_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:18,781 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796278402_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:18,781 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796278402_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:18,782 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796278402_0001_00_000002 given task: attempt_1452796278402_0001_1_01_000000_0
2016-01-14 18:31:18,782 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796278402_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796278402_0001_1_01_000000_0
2016-01-14 18:31:18 Starting to run new task attempt: attempt_1452796278402_0001_1_01_000000_0
2016-01-14 18:31:18,783 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796278402_0001_1_01_000000_0
2016-01-14 18:31:18,783 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:18,783 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:18,783 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:18,783 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:18,783 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:18,783 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:18,783 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386] every/(1/1) ...blyplatform/-1_UNKNOWN_-1, VertexName: CE2A4478E0B04F1D8EE03D1335EE9234, VertexParallelism: 1, TaskAttemptID:attempt_1452796278402_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BDAD36BEBD0541D89B5B2C0C91271A43, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2C328B3CF3AA471C88D2E28DD7126EE3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:18,784 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:18,791 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796278402_0001_1_01_000000_0
2016-01-14 18:31:18,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:18,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:18,791 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796278402_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:18,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BDAD36BEBD0541D89B5B2C0C91271A43, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:18,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2C328B3CF3AA471C88D2E28DD7126EE3, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:18,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:18,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:18,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BDAD36BEBD0541D89B5B2C0C91271A43
2016-01-14 18:31:18,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:18,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2C328B3CF3AA471C88D2E28DD7126EE3
2016-01-14 18:31:18,801 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:18,801 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:18,800 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:18,808 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:18,808 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BDAD36BEBD0541D89B5B2C0C91271A43
2016-01-14 18:31:18,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BDAD36BEBD0541D89B5B2C0C91271A43
2016-01-14 18:31:18,802 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2C328B3CF3AA471C88D2E28DD7126EE3
2016-01-14 18:31:18,810 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CE2A4478E0B04F1D8EE03D1335EE9234, ordinal: 1
2016-01-14 18:31:18,811 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE2A4478E0B04F1D8EE03D1335EE9234, mem on start (mb), free: 133, total: 490, max: 672
2016-01-14 18:31:18,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:18,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:18,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:18,822 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:18,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2C328B3CF3AA471C88D2E28DD7126EE3
2016-01-14 18:31:18,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:18,822 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:18,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:18,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:18,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:18,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:18,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:18,824 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BDAD36BEBD0541D89B5B2C0C91271A43, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:18,824 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2C328B3CF3AA471C88D2E28DD7126EE3, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:18,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:18,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BDAD36BEBD0541D89B5B2C0C91271A43 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:18,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BDAD36BEBD0541D89B5B2C0C91271A43
2016-01-14 18:31:18,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:18,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:18,825 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:18,825 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:18,826 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BDAD36BEBD0541D89B5B2C0C91271A43 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:18,826 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:18,826 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:18,826 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:18,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BDAD36BEBD0541D89B5B2C0C91271A43
2016-01-14 18:31:18,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:18,833 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796278402_0001_1_01_000000_0
2016-01-14 18:31:18,838 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:18,838 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796278402_0001_1_00_000000_0_10033, runDuration: 58706, ]
2016-01-14 18:31:18,839 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 5DDDC385431044578A86EA3CABE1AB38
2016-01-14 18:31:18,839 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1"], id: 2C328B3CF3AA471C88D2E28DD7126EE3
2016-01-14 18:31:18,840 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_-1"]
2016-01-14 18:31:18,840 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 5DDDC385431044578A86EA3CABE1AB38, for 1 inputs
2016-01-14 18:31:18,841 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796278402_0001_1_00_000000_0_10033] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796278841, TimeTaken=2, Rate=0.14 MB/s
2016-01-14 18:31:18,841 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BDAD36BEBD0541D89B5B2C0C91271A43
2016-01-14 18:31:18,843 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:18,843 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BDAD36BEBD0541D89B5B2C0C91271A43] #6 in 4ms
2016-01-14 18:31:18,843 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BDAD36BEBD0541D89B5B2C0C91271A43] #6, status:false, isInterrupted:false
2016-01-14 18:31:18,844 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:18,844 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:18,844 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:18,845 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:18,845 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:18,845 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BDAD36BEBD0541D89B5B2C0C91271A43
2016-01-14 18:31:18,847 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:18,847 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CE2A4478E0B04F1D8EE03D1335EE9234, all 1 inputs ready in: 00:00:00.007
2016-01-14 18:31:18,849 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@52e38b81; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@58dbecaa
2016-01-14 18:31:18,859 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:18,860 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:18,861 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796278402_0001_1_01_000000_0
2016-01-14 18:31:18,861 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796278402_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:18,862 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796278402_0001_1_01_000000_0
2016-01-14 18:31:18,862 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CE2A4478E0B04F1D8EE03D1335EE9234, mem on close (mb), free: 128, total: 490, max: 672
2016-01-14 18:31:18,863 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BDAD36BEBD0541D89B5B2C0C91271A43
2016-01-14 18:31:18,864 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:18,864 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:18,864 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:18,865 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:18,865 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:18 Completed running task attempt: attempt_1452796278402_0001_1_01_000000_0
2016-01-14 18:31:18,868 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796278402_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:18,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796278848, Process_Duration=14, Process_End_Time=1452796278862, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:18,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CE2A4478E0B04F1D8EE03D1335EE9234, taskAttemptId=attempt_1452796278402_0001_1_01_000000_0, startTime=1452796278780, finishTime=1452796278870, timeTaken=90, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796278848, Process_Duration=14, Process_End_Time=1452796278862, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:18,871 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796278402_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:18,871 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:18,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:18,871 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796278402_0001_1_01_000000_0
2016-01-14 18:31:18,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:TASK_FINISHED]: vertexName=CE2A4478E0B04F1D8EE03D1335EE9234, taskId=task_1452796278402_0001_1_01_000000, startTime=1452796278780, finishTime=1452796278872, timeTaken=92, status=SUCCEEDED, successfulAttemptID=attempt_1452796278402_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796278848, Process_Duration=14, Process_End_Time=1452796278862, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:18,873 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:18,873 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796278402_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:18,874 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:18,874 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:18,874 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796278402_0001_00_000002
2016-01-14 18:31:18,874 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234] : 1
2016-01-14 18:31:18,874 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234]
2016-01-14 18:31:18,876 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:18,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:VERTEX_FINISHED]: vertexName=CE2A4478E0B04F1D8EE03D1335EE9234, vertexId=vertex_1452796278402_0001_1_01, initRequestedTime=1452796278606, initedTime=1452796278619, startRequestedTime=1452796278644, startedTime=1452796278644, finishTime=1452796278874, timeTaken=230, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796278848, Process_Duration=14, Process_End_Time=1452796278862, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796278780, firstTasksToStart=[ task_1452796278402_0001_1_01_000000 ], lastTaskFinishTime=1452796278870, lastTasksToFinish=[ task_1452796278402_0001_1_01_000000 ], minTaskDuration=90, maxTaskDuration=90, avgTaskDuration=90.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796278402_0001_1_01_000000 ], longestDurationTasks=[ task_1452796278402_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:18,881 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:18,881 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:18,882 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:18,882 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796278402_0001_00_000002, stoppedTime=1452796278882, exitStatus=0
2016-01-14 18:31:18,883 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:18,883 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:18,883 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796278402_0001_1
2016-01-14 18:31:18,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796278402_0001_1
2016-01-14 18:31:18,884 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2C328B3CF3AA471C88D2E28DD7126EE3 for vertex: vertex_1452796278402_0001_1_01 [CE2A4478E0B04F1D8EE03D1335EE9234]
2016-01-14 18:31:18,887 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796278402_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:18,888 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796278402_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:18,888 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796278402_0001_00_000002, Exiting
2016-01-14 18:31:18,888 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796278402_0001_00_000002
2016-01-14 18:31:18,888 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796278402_0001_00_000002
2016-01-14 18:31:18,890 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796278402_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:18,890 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796278402_0001_00_000002 completed successfully
2016-01-14 18:31:18,893 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796278402_0001_1_00 [BDAD36BEBD0541D89B5B2C0C91271A43]
2016-01-14 18:31:18,896 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796278402_0001_1, queueSize=0
2016-01-14 18:31:18,896 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:18,897 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796278402_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796278402_0001_1, startTime=1452796278600, finishTime=1452796278893, timeTaken=293, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1028653056, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=17, MERGE_PHASE_TIME=19, FIRST_EVENT_RECEIVED=13, LAST_EVENT_RECEIVED=13, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592557573, Process_Duration=19, Process_End_Time=2905592557592, Read_Duration=5, Tuples_Read=40, Tuples_Written=40, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:18,897 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796278402_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:18,898 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796278402_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:18,898 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796278402_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:18 Completed Dag: dag_1452796278402_0001_1
2016-01-14 18:31:18,899 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796278402_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:18,899 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796278402_0001_1_post
2016-01-14 18:31:18,900 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:18,900 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:18,900 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796278402_0001_00_000002
2016-01-14 18:31:18,907 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_UNKNOWN_-1, applicationId=application_1452796278402_0001
2016-01-14 18:31:18,912 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:18,912 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:18,912 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:18,912 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:18,912 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:18,912 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:18,913 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:18,914 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:18,915 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:18,915 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:18,915 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:18,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:18,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:18,922 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/57ECAA82E39340AC84C4689BE080AC84/D24737097C3A4045A33A70225031D386/.tez/application_1452796278402_0001
2016-01-14 18:31:18,923 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.543
2016-01-14 18:31:19,036 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_-1
2016-01-14 18:31:19,074 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:19,075 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:19,128 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.011
2016-01-14 18:31:19,129 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.012
2016-01-14 18:31:19,129 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:19,130 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:19,130 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_'field'
2016-01-14 18:31:19,173 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:19,173 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:19,240 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 18:31:19,258 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.038
2016-01-14 18:31:19,259 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:19,259 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:19,260 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:19,286 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:19,297 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS
2016-01-14 18:31:19,370 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:19,371 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:19,371 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS"]
2016-01-14 18:31:19,372 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:19,372 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:19,372 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:19,373 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/-1_UNKNOWN_RESULTS
2016-01-14 18:31:19,374 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:19,390 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:19,394 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:19,394 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:19,394 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:19,404 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE/.tez/application_1452796279394_0001 doesn't exist and is created
2016-01-14 18:31:19,418 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:19,418 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:19,419 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE/.tez/application_1452796279394_0001
2016-01-14 18:31:19,421 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796279394_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:19,432 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:19,433 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:19,435 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:19,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:19,435 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE/.tez/application_1452796279394_0001/localmode-log-dir
2016-01-14 18:31:19,435 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE/.tez/application_1452796279394_0001/localmode-log-dir/history.txt.appattempt_1452796279394_0001_000000, maxErrors=10
2016-01-14 18:31:19,435 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:19,436 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796279394_0001_000000, appSubmitTime=1452796279420, launchTime=1452796279420
2016-01-14 18:31:19,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:19,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:19,443 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:19,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:19,448 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:60504
2016-01-14 18:31:19,449 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:19,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796279394_0001_000000, startTime=1452796279451
2016-01-14 18:31:19,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:19,451 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:19,472 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:19,473 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:19,519 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:19,519 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:19,519 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1452796279394_0001, dagName=[3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-01-14 18:31:19,541 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-01-14 18:31:19,543 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796279394_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE/.tez/application_1452796279394_0001/localmode-log-dir/dag_1452796279394_0001_1.dot
2016-01-14 18:31:19,544 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE/.tez/application_1452796279394_0001/localmode-log-dir/dag_1452796279394_0001_1-tez-dag.pb.txt
2016-01-14 18:31:19,559 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796279394_0001_1
2016-01-14 18:31:19,559 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:19,559 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-01-14 18:31:19 Running Dag: dag_1452796279394_0001_1
2016-01-14 18:31:19,570 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796279394_0001_1, submitTime=1452796279542
2016-01-14 18:31:19,570 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:19,571 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AC08F843588F4831ABB8DF675DADC72B
2016-01-14 18:31:19,571 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:19,571 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796279394_0001_1, initTime=1452796279570
2016-01-14 18:31:19,571 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:19,572 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796279394_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:19,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:DAG_STARTED]: dagID=dag_1452796279394_0001_1, startTime=1452796279572
2016-01-14 18:31:19,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:19,573 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:19,573 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796279394_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:19,573 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FE028375C37D47949FBEBAC4973949FD : {928EB46CDAF941938986DFC78FF32E42={InputName=928EB46CDAF941938986DFC78FF32E42}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:19,573 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 928EB46CDAF941938986DFC78FF32E42, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:19,573 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD]
2016-01-14 18:31:19,574 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD]
2016-01-14 18:31:19,574 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD]
2016-01-14 18:31:19,574 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD]. Starting root input initializers: 1
2016-01-14 18:31:19,576 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:19,577 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B]
2016-01-14 18:31:19,578 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1452796279394_0001, dagName=[3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE] every/(1/1) ...atform/-1_UNKNOWN_RESULTS
2016-01-14 18:31:19,577 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 928EB46CDAF941938986DFC78FF32E42 on vertex vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD]
2016-01-14 18:31:19,581 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:19,581 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B]
2016-01-14 18:31:19,581 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B]
2016-01-14 18:31:19,581 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B]
2016-01-14 18:31:19,581 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CA2618D9D25D4A3E82864DFC5FF13C1E, vertexId=vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:19,582 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CA2618D9D25D4A3E82864DFC5FF13C1E, vertexId=vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B]
2016-01-14 18:31:19,583 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:19,584 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 928EB46CDAF941938986DFC78FF32E42 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:19,584 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:19,585 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CA2618D9D25D4A3E82864DFC5FF13C1E, vertexName=AC08F843588F4831ABB8DF675DADC72B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:19,585 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:19,586 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CA2618D9D25D4A3E82864DFC5FF13C1E, vertexId=vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B]
2016-01-14 18:31:19,587 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:19,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AC08F843588F4831ABB8DF675DADC72B, vertexId=vertex_1452796279394_0001_1_01, initRequestedTime=1452796279577, initedTime=1452796279587, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:19,588 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:19,590 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:19,591 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 928EB46CDAF941938986DFC78FF32E42 on vertex vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD]
2016-01-14 18:31:19,591 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:19,591 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {928EB46CDAF941938986DFC78FF32E42=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:19,592 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD] parallelism set to 1
2016-01-14 18:31:19,592 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:19,593 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796279394_0001_1_00, eventCount=1
2016-01-14 18:31:19,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:19,594 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FE028375C37D47949FBEBAC4973949FD, vertexId=vertex_1452796279394_0001_1_00, initRequestedTime=1452796279573, initedTime=1452796279593, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:19,594 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD] with distanceFromRoot: 0
2016-01-14 18:31:19,594 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:19,595 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FE028375C37D47949FBEBAC4973949FD
2016-01-14 18:31:19,595 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD]
2016-01-14 18:31:19,596 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796279394_0001_1_00, startRequestedTime=1452796279595, startedTime=1452796279595
2016-01-14 18:31:19,596 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:19,596 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:19,597 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:19,597 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:TASK_STARTED]: vertexName=FE028375C37D47949FBEBAC4973949FD, taskId=task_1452796279394_0001_1_00_000000, scheduledTime=1452796279596, launchTime=1452796279596
2016-01-14 18:31:19,597 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796279394_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:19,597 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796279394_0001_1_00 for vertex: vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B] numStartedSources: 1 numSources: 1
2016-01-14 18:31:19,598 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B] with distanceFromRoot: 1
2016-01-14 18:31:19,598 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD]
2016-01-14 18:31:19,598 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FE028375C37D47949FBEBAC4973949FD in vertex: AC08F843588F4831ABB8DF675DADC72B
2016-01-14 18:31:19,599 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: AC08F843588F4831ABB8DF675DADC72B with 1 source tasks and 1 pending tasks
2016-01-14 18:31:19,599 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796279394_0001_1_01, startRequestedTime=1452796279598, startedTime=1452796279598
2016-01-14 18:31:19,599 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:19,599 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:19,600 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: FE028375C37D47949FBEBAC4973949FD, VertexParallelism: 1, TaskAttemptID:attempt_1452796279394_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=928EB46CDAF941938986DFC78FF32E42, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AC08F843588F4831ABB8DF675DADC72B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:19,600 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796279394_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:19,601 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:19,602 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:19,602 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:19,602 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:19,603 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796279394_0001_00_000001
2016-01-14 18:31:19,603 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:19,605 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796279394_0001_00_000001
2016-01-14 18:31:19,605 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:19,608 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:19,609 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:19,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796279394_0001_00_000001, launchTime=1452796279609
2016-01-14 18:31:19,610 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796279394_0001_00_000001
2016-01-14 18:31:19,611 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796279394_0001_1_00_000000_0] to container: [container_1452796279394_0001_00_000001]
2016-01-14 18:31:19,612 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:19,613 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796279394_0001_1_00_000000_0] started. Is using containerId: [container_1452796279394_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:19,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:19,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FE028375C37D47949FBEBAC4973949FD, taskAttemptId=attempt_1452796279394_0001_1_00_000000_0, startTime=1452796279613, containerId=container_1452796279394_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796279394_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:19,613 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796279394_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:19,613 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796279394_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:19,615 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796279394_0001_00_000001 given task: attempt_1452796279394_0001_1_00_000000_0
2016-01-14 18:31:19,615 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796279394_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796279394_0001_1_00_000000_0
2016-01-14 18:31:19 Starting to run new task attempt: attempt_1452796279394_0001_1_00_000000_0
2016-01-14 18:31:19,616 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796279394_0001_1_00_000000_0
2016-01-14 18:31:19,616 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:19,617 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:19,617 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:19,617 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:19,617 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:19,617 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:19,618 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: FE028375C37D47949FBEBAC4973949FD, VertexParallelism: 1, TaskAttemptID:attempt_1452796279394_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=928EB46CDAF941938986DFC78FF32E42, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AC08F843588F4831ABB8DF675DADC72B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:19,618 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:19,619 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796279394_0001_1_00_000000_0
2016-01-14 18:31:19,619 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796279394_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:19,619 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:19,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:19,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=928EB46CDAF941938986DFC78FF32E42, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:19,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AC08F843588F4831ABB8DF675DADC72B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:19,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:19,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:19,620 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:19,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 928EB46CDAF941938986DFC78FF32E42
2016-01-14 18:31:19,622 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 928EB46CDAF941938986DFC78FF32E42
2016-01-14 18:31:19,622 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796279394_0001, with dag id: dag_1452796279394_0001_1
2016-01-14 18:31:19,628 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:19,629 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:19,629 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 928EB46CDAF941938986DFC78FF32E42
2016-01-14 18:31:19,629 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 928EB46CDAF941938986DFC78FF32E42
2016-01-14 18:31:19,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AC08F843588F4831ABB8DF675DADC72B
2016-01-14 18:31:19,632 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:19,632 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: AC08F843588F4831ABB8DF675DADC72B
2016-01-14 18:31:19,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AC08F843588F4831ABB8DF675DADC72B
2016-01-14 18:31:19,635 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FE028375C37D47949FBEBAC4973949FD, ordinal: 0
2016-01-14 18:31:19,635 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE028375C37D47949FBEBAC4973949FD, mem on start (mb), free: 215, total: 491, max: 672
2016-01-14 18:31:19,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:19,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:19,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:19,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:19,636 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:19,637 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:19,637 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:19,637 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:19,637 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:19,638 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:19,638 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 928EB46CDAF941938986DFC78FF32E42, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:19,638 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AC08F843588F4831ABB8DF675DADC72B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:19,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:19,639 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 928EB46CDAF941938986DFC78FF32E42 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:19,639 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 928EB46CDAF941938986DFC78FF32E42
2016-01-14 18:31:19,639 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:19,639 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 928EB46CDAF941938986DFC78FF32E42
2016-01-14 18:31:19,639 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:19,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:19,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:19,640 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796279394_0001_1_00_000000_0
2016-01-14 18:31:19,640 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:19,641 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@126717c8
2016-01-14 18:31:19,642 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@126717c8
2016-01-14 18:31:19,642 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:19,642 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:19,645 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:19,646 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 928EB46CDAF941938986DFC78FF32E42
2016-01-14 18:31:19,646 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: D39F959D4F2E4E4EAB1737A7739277A6
2016-01-14 18:31:19,647 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] D39F959D4F2E4E4EAB1737A7739277A6
2016-01-14 18:31:19,648 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@643cca36; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@52ffd2a3; comparator=cascading.tuple.hadoop.util.TupleComparator@195df5e7; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:19,648 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:19,648 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:19,649 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:19,662 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:19,662 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:19,663 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:19,663 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:19,663 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:19,664 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:19,664 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:19,665 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FE028375C37D47949FBEBAC4973949FD, all 1 inputs ready in: 00:00:00.001
2016-01-14 18:31:19,668 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796279394_0001_1_00_000000_0
2016-01-14 18:31:19,668 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE028375C37D47949FBEBAC4973949FD, mem on close (mb), free: 113, total: 491, max: 672
2016-01-14 18:31:19,668 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:19,669 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:19,669 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:19,669 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:19,671 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:19 Completed running task attempt: attempt_1452796279394_0001_1_00_000000_0
2016-01-14 18:31:19,675 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796279394_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:19,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796279665, Process_Duration=3, Process_End_Time=1452796279668, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:19,676 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:19,676 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:19,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796279394_0001_1_00, eventCount=1
2016-01-14 18:31:19,677 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796279394_0001_00_000001
2016-01-14 18:31:19,677 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:19,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FE028375C37D47949FBEBAC4973949FD, taskAttemptId=attempt_1452796279394_0001_1_00_000000_0, startTime=1452796279613, finishTime=1452796279677, timeTaken=64, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796279665, Process_Duration=3, Process_End_Time=1452796279668, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:19,677 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796279394_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:19,678 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796279394_0001_1_00_000000_0
2016-01-14 18:31:19,678 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:19,678 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:19,679 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:TASK_FINISHED]: vertexName=FE028375C37D47949FBEBAC4973949FD, taskId=task_1452796279394_0001_1_00_000000, startTime=1452796279613, finishTime=1452796279678, timeTaken=65, status=SUCCEEDED, successfulAttemptID=attempt_1452796279394_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796279665, Process_Duration=3, Process_End_Time=1452796279668, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:19,679 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796279394_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:19,679 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD] : 1
2016-01-14 18:31:19,679 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD]
2016-01-14 18:31:19,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:19,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:VERTEX_FINISHED]: vertexName=FE028375C37D47949FBEBAC4973949FD, vertexId=vertex_1452796279394_0001_1_00, initRequestedTime=1452796279573, initedTime=1452796279593, startRequestedTime=1452796279595, startedTime=1452796279595, finishTime=1452796279679, timeTaken=84, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796279665, Process_Duration=3, Process_End_Time=1452796279668, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796279613, firstTasksToStart=[ task_1452796279394_0001_1_00_000000 ], lastTaskFinishTime=1452796279677, lastTasksToFinish=[ task_1452796279394_0001_1_00_000000 ], minTaskDuration=64, maxTaskDuration=64, avgTaskDuration=64.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796279394_0001_1_00_000000 ], longestDurationTasks=[ task_1452796279394_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:19,681 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:19,681 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:19,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796279394_0001_00_000001, stoppedTime=1452796279682, exitStatus=0
2016-01-14 18:31:19,682 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:19,682 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:19,682 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B] attempt: attempt_1452796279394_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:19,682 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AC08F843588F4831ABB8DF675DADC72B
2016-01-14 18:31:19,682 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B]
2016-01-14 18:31:19,683 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:19,683 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796279394_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:19,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:19,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:TASK_STARTED]: vertexName=AC08F843588F4831ABB8DF675DADC72B, taskId=task_1452796279394_0001_1_01_000000, scheduledTime=1452796279683, launchTime=1452796279683
2016-01-14 18:31:19,684 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796279394_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:19,684 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:19,684 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B]
2016-01-14 18:31:19,685 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: AC08F843588F4831ABB8DF675DADC72B, VertexParallelism: 1, TaskAttemptID:attempt_1452796279394_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FE028375C37D47949FBEBAC4973949FD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CA2618D9D25D4A3E82864DFC5FF13C1E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:19,685 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796279394_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:19,685 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:19,687 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796279394_0001_00_000002
2016-01-14 18:31:19,687 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:19,687 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796279394_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:19,688 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796279394_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:19,688 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796279394_0001_00_000001, Exiting
2016-01-14 18:31:19,688 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796279394_0001_00_000001
2016-01-14 18:31:19,688 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796279394_0001_00_000001
2016-01-14 18:31:19,688 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796279394_0001_00_000002
2016-01-14 18:31:19,690 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:19,691 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796279394_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:19,691 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796279394_0001_00_000001 completed successfully
2016-01-14 18:31:19,691 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:19,692 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:19,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:19,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796279394_0001_00_000002, launchTime=1452796279692
2016-01-14 18:31:19,693 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796279394_0001_00_000002
2016-01-14 18:31:19,694 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796279394_0001_1_01_000000_0] to container: [container_1452796279394_0001_00_000002]
2016-01-14 18:31:19,694 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:19,695 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796279394_0001_00_000002 given task: attempt_1452796279394_0001_1_01_000000_0
2016-01-14 18:31:19,695 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796279394_0001_1_01_000000_0] started. Is using containerId: [container_1452796279394_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:19,695 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796279394_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796279394_0001_1_01_000000_0
2016-01-14 18:31:19,695 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AC08F843588F4831ABB8DF675DADC72B, taskAttemptId=attempt_1452796279394_0001_1_01_000000_0, startTime=1452796279695, containerId=container_1452796279394_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796279394_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:19,695 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:19,695 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796279394_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:19,696 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796279394_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:19 Starting to run new task attempt: attempt_1452796279394_0001_1_01_000000_0
2016-01-14 18:31:19,696 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796279394_0001_1_01_000000_0
2016-01-14 18:31:19,696 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:19,696 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:19,696 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:19,697 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:19,697 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:19,698 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:19,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE] every/(1/1) ...atform/-1_UNKNOWN_RESULTS, VertexName: AC08F843588F4831ABB8DF675DADC72B, VertexParallelism: 1, TaskAttemptID:attempt_1452796279394_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FE028375C37D47949FBEBAC4973949FD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CA2618D9D25D4A3E82864DFC5FF13C1E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:19,698 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:19,699 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796279394_0001_1_01_000000_0
2016-01-14 18:31:19,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:19,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:19,700 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796279394_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:19,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:19,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FE028375C37D47949FBEBAC4973949FD, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:19,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CA2618D9D25D4A3E82864DFC5FF13C1E, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:19,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:19,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FE028375C37D47949FBEBAC4973949FD
2016-01-14 18:31:19,701 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:19,702 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CA2618D9D25D4A3E82864DFC5FF13C1E
2016-01-14 18:31:19,702 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:19,702 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CA2618D9D25D4A3E82864DFC5FF13C1E
2016-01-14 18:31:19,703 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:19,703 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:19,703 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FE028375C37D47949FBEBAC4973949FD
2016-01-14 18:31:19,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FE028375C37D47949FBEBAC4973949FD
2016-01-14 18:31:19,708 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:19,712 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:19,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CA2618D9D25D4A3E82864DFC5FF13C1E
2016-01-14 18:31:19,716 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AC08F843588F4831ABB8DF675DADC72B, ordinal: 1
2016-01-14 18:31:19,716 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC08F843588F4831ABB8DF675DADC72B, mem on start (mb), free: 106, total: 491, max: 672
2016-01-14 18:31:19,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:19,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:19,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:19,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:19,718 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:19,718 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:19,718 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:19,719 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:19,719 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:19,719 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:19,719 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CA2618D9D25D4A3E82864DFC5FF13C1E, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:19,720 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FE028375C37D47949FBEBAC4973949FD, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:19,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:19,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FE028375C37D47949FBEBAC4973949FD being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:19,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FE028375C37D47949FBEBAC4973949FD
2016-01-14 18:31:19,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:19,721 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:19,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:19,722 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:19,724 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FE028375C37D47949FBEBAC4973949FD with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:19,724 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:19,724 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:19,724 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:19,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FE028375C37D47949FBEBAC4973949FD
2016-01-14 18:31:19,726 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:19,726 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796279394_0001_1_01_000000_0
2016-01-14 18:31:19,728 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796279394_0001_1_00_000000_0_10039, runDuration: 42782, ]
2016-01-14 18:31:19,730 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796279394_0001_1_00_000000_0_10039] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796279729, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:19,730 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FE028375C37D47949FBEBAC4973949FD
2016-01-14 18:31:19,730 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:19,730 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FE028375C37D47949FBEBAC4973949FD] #7 in 1ms
2016-01-14 18:31:19,730 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:19,731 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: D39F959D4F2E4E4EAB1737A7739277A6
2016-01-14 18:31:19,731 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS"], id: CA2618D9D25D4A3E82864DFC5FF13C1E
2016-01-14 18:31:19,732 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_RESULTS"]
2016-01-14 18:31:19,732 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FE028375C37D47949FBEBAC4973949FD] #7, status:false, isInterrupted:false
2016-01-14 18:31:19,732 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] D39F959D4F2E4E4EAB1737A7739277A6, for 1 inputs
2016-01-14 18:31:19,733 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:19,733 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:19,733 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:19,734 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:19,734 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:19,735 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AC08F843588F4831ABB8DF675DADC72B, all 1 inputs ready in: 00:00:00.002
2016-01-14 18:31:19,735 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FE028375C37D47949FBEBAC4973949FD
2016-01-14 18:31:19,737 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3dec6eac; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6d55fdf6
2016-01-14 18:31:19,737 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:19,740 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:19,741 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:19,741 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796279394_0001_1_01_000000_0
2016-01-14 18:31:19,741 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796279394_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:19,742 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796279394_0001_1_01_000000_0
2016-01-14 18:31:19,742 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AC08F843588F4831ABB8DF675DADC72B, mem on close (mb), free: 102, total: 491, max: 672
2016-01-14 18:31:19,742 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FE028375C37D47949FBEBAC4973949FD
2016-01-14 18:31:19,743 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:19,743 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:19,743 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:19,743 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:19,743 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:19 Completed running task attempt: attempt_1452796279394_0001_1_01_000000_0
2016-01-14 18:31:19,744 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796279394_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:19,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796279735, Process_Duration=7, Process_End_Time=1452796279742, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:19,745 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:19,745 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:19,746 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AC08F843588F4831ABB8DF675DADC72B, taskAttemptId=attempt_1452796279394_0001_1_01_000000_0, startTime=1452796279695, finishTime=1452796279745, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796279735, Process_Duration=7, Process_End_Time=1452796279742, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:19,746 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:19,746 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796279394_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:19,746 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796279394_0001_00_000002
2016-01-14 18:31:19,746 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796279394_0001_1_01_000000_0
2016-01-14 18:31:19,747 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:TASK_FINISHED]: vertexName=AC08F843588F4831ABB8DF675DADC72B, taskId=task_1452796279394_0001_1_01_000000, startTime=1452796279695, finishTime=1452796279746, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1452796279394_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796279735, Process_Duration=7, Process_End_Time=1452796279742, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:19,747 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796279394_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:19,748 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B] : 1
2016-01-14 18:31:19,748 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B]
2016-01-14 18:31:19,749 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:19,749 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:19,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:19,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:VERTEX_FINISHED]: vertexName=AC08F843588F4831ABB8DF675DADC72B, vertexId=vertex_1452796279394_0001_1_01, initRequestedTime=1452796279577, initedTime=1452796279587, startRequestedTime=1452796279598, startedTime=1452796279598, finishTime=1452796279748, timeTaken=150, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514850816, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796279735, Process_Duration=7, Process_End_Time=1452796279742, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796279695, firstTasksToStart=[ task_1452796279394_0001_1_01_000000 ], lastTaskFinishTime=1452796279745, lastTasksToFinish=[ task_1452796279394_0001_1_01_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796279394_0001_1_01_000000 ], longestDurationTasks=[ task_1452796279394_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:19,750 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:19,751 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:19,752 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:19,752 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796279394_0001_1
2016-01-14 18:31:19,753 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796279394_0001_1
2016-01-14 18:31:19,753 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796279394_0001_1_00 [FE028375C37D47949FBEBAC4973949FD]
2016-01-14 18:31:19,753 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: CA2618D9D25D4A3E82864DFC5FF13C1E for vertex: vertex_1452796279394_0001_1_01 [AC08F843588F4831ABB8DF675DADC72B]
2016-01-14 18:31:19,761 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796279394_0001_1, queueSize=0
2016-01-14 18:31:19,762 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:19,763 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796279394_0001_1, startTime=1452796279572, finishTime=1452796279759, timeTaken=187, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1029701632, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592559400, Process_Duration=10, Process_End_Time=2905592559410, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:19,763 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796279394_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:19,764 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796279394_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:19,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796279394_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796279394_0001_00_000002, stoppedTime=1452796279764, exitStatus=0
2016-01-14 18:31:19,766 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:19,764 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:19 Completed Dag: dag_1452796279394_0001_1
2016-01-14 18:31:19,768 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796279394_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:19,769 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796279394_0001_1_post
2016-01-14 18:31:19,769 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:19,769 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:19,770 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796279394_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:19,770 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:19,772 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_UNKNOWN_RESULTS, applicationId=application_1452796279394_0001
2016-01-14 18:31:19,775 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:19,775 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:19,775 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:19,775 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:19,775 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:19,776 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:19,776 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:19,777 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:19,777 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:19,777 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796279394_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:19,777 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796279394_0001_00_000002: 31 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:19,778 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796279394_0001_00_000002, Exiting
2016-01-14 18:31:19,778 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796279394_0001_00_000002
2016-01-14 18:31:19,778 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796279394_0001_00_000002
2016-01-14 18:31:19,779 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:19,780 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796279394_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:19,780 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796279394_0001_00_000002 completed successfully
2016-01-14 18:31:19,780 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:19,780 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796279394_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:19,780 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:19,780 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:19,786 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3BC09A97CF014B20BAABEC4DC8E83632/FEC7E35CA6E140FDAE19A3B69E7C96AE/.tez/application_1452796279394_0001
2016-01-14 18:31:19,786 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.416
2016-01-14 18:31:19,867 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_RESULTS
2016-01-14 18:31:19,894 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:19,895 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:19,936 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 18:31:19,945 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.022
2016-01-14 18:31:19,946 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:19,946 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:19,946 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:19,965 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:19,972 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL
2016-01-14 18:31:20,037 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:20,037 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:20,037 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL"]
2016-01-14 18:31:20,038 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:20,038 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:20,038 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:20,039 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-01-14 18:31:20,039 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:20,053 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:20,058 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:20,058 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:20,059 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:20,068 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9/.tez/application_1452796280058_0001 doesn't exist and is created
2016-01-14 18:31:20,083 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:20,083 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:20,084 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9/.tez/application_1452796280058_0001
2016-01-14 18:31:20,086 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796280058_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:20,097 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:20,098 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:20,100 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:20,100 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:20,100 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9/.tez/application_1452796280058_0001/localmode-log-dir
2016-01-14 18:31:20,100 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9/.tez/application_1452796280058_0001/localmode-log-dir/history.txt.appattempt_1452796280058_0001_000000, maxErrors=10
2016-01-14 18:31:20,100 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:20,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796280058_0001_000000, appSubmitTime=1452796280085, launchTime=1452796280086
2016-01-14 18:31:20,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:20,108 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:32770
2016-01-14 18:31:20,109 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:20,110 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:20,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:20,111 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:20,114 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796280058_0001_000000, startTime=1452796280114
2016-01-14 18:31:20,114 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:20,114 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:20,183 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:20,184 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:20,184 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1452796280058_0001, dagName=[F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-01-14 18:31:20,202 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-01-14 18:31:20,204 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796280058_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9/.tez/application_1452796280058_0001/localmode-log-dir/dag_1452796280058_0001_1.dot
2016-01-14 18:31:20,204 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9/.tez/application_1452796280058_0001/localmode-log-dir/dag_1452796280058_0001_1-tez-dag.pb.txt
2016-01-14 18:31:20,220 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796280058_0001_1
2016-01-14 18:31:20,220 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:20,220 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-01-14 18:31:20 Running Dag: dag_1452796280058_0001_1
2016-01-14 18:31:20,230 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796280058_0001_1, submitTime=1452796280202
2016-01-14 18:31:20,230 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:20,231 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A11731EDB03E4D6B82B29DFC93C6D189
2016-01-14 18:31:20,231 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:20,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:20,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796280058_0001_1, initTime=1452796280230
2016-01-14 18:31:20,232 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796280058_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:20,233 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:DAG_STARTED]: dagID=dag_1452796280058_0001_1, startTime=1452796280233
2016-01-14 18:31:20,233 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:20,233 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:20,233 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796280058_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:20,233 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FDCE8BFF601B49C29BAC1835C168CCD0 : {2702CC0160CE43DEB62A8E93357E14B8={InputName=2702CC0160CE43DEB62A8E93357E14B8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:20,234 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2702CC0160CE43DEB62A8E93357E14B8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:20,234 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0]
2016-01-14 18:31:20,234 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0]
2016-01-14 18:31:20,234 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0]
2016-01-14 18:31:20,234 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1452796280058_0001, dagName=[F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL
2016-01-14 18:31:20,234 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0]. Starting root input initializers: 1
2016-01-14 18:31:20,235 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:20,235 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189]
2016-01-14 18:31:20,237 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2702CC0160CE43DEB62A8E93357E14B8 on vertex vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0]
2016-01-14 18:31:20,238 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:20,238 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189]
2016-01-14 18:31:20,238 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189]
2016-01-14 18:31:20,238 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189]
2016-01-14 18:31:20,238 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=548ED704E7E346469468760B365DC75B, vertexId=vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:20,238 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=548ED704E7E346469468760B365DC75B, vertexId=vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189]
2016-01-14 18:31:20,241 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=548ED704E7E346469468760B365DC75B, vertexName=A11731EDB03E4D6B82B29DFC93C6D189, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:20,241 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=548ED704E7E346469468760B365DC75B, vertexId=vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189]
2016-01-14 18:31:20,242 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:20,243 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:20,243 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2702CC0160CE43DEB62A8E93357E14B8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:20,243 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A11731EDB03E4D6B82B29DFC93C6D189, vertexId=vertex_1452796280058_0001_1_01, initRequestedTime=1452796280235, initedTime=1452796280242, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:20,243 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:20,244 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:20,244 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:20,246 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:20,246 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2702CC0160CE43DEB62A8E93357E14B8 on vertex vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0]
2016-01-14 18:31:20,246 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:20,247 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2702CC0160CE43DEB62A8E93357E14B8=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:20,247 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0] parallelism set to 1
2016-01-14 18:31:20,247 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:20,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796280058_0001_1_00, eventCount=1
2016-01-14 18:31:20,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:20,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FDCE8BFF601B49C29BAC1835C168CCD0, vertexId=vertex_1452796280058_0001_1_00, initRequestedTime=1452796280233, initedTime=1452796280248, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:20,249 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0] with distanceFromRoot: 0
2016-01-14 18:31:20,249 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:20,250 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FDCE8BFF601B49C29BAC1835C168CCD0
2016-01-14 18:31:20,250 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0]
2016-01-14 18:31:20,250 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:20,251 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796280058_0001_1_00, startRequestedTime=1452796280249, startedTime=1452796280249
2016-01-14 18:31:20,251 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:20,251 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:20,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:TASK_STARTED]: vertexName=FDCE8BFF601B49C29BAC1835C168CCD0, taskId=task_1452796280058_0001_1_00_000000, scheduledTime=1452796280251, launchTime=1452796280251
2016-01-14 18:31:20,252 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280058_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:20,252 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796280058_0001_1_00 for vertex: vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189] numStartedSources: 1 numSources: 1
2016-01-14 18:31:20,252 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189] with distanceFromRoot: 1
2016-01-14 18:31:20,253 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0]
2016-01-14 18:31:20,253 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FDCE8BFF601B49C29BAC1835C168CCD0 in vertex: A11731EDB03E4D6B82B29DFC93C6D189
2016-01-14 18:31:20,253 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A11731EDB03E4D6B82B29DFC93C6D189 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:20,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:20,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796280058_0001_1_01, startRequestedTime=1452796280253, startedTime=1452796280253
2016-01-14 18:31:20,255 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:20,255 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: FDCE8BFF601B49C29BAC1835C168CCD0, VertexParallelism: 1, TaskAttemptID:attempt_1452796280058_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2702CC0160CE43DEB62A8E93357E14B8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A11731EDB03E4D6B82B29DFC93C6D189, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:20,256 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280058_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:20,256 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:20,256 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796280058_0001, with dag id: dag_1452796280058_0001_1
2016-01-14 18:31:20,257 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:20,257 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:20,258 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:20,258 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796280058_0001_00_000001
2016-01-14 18:31:20,258 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:20,262 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796280058_0001_00_000001
2016-01-14 18:31:20,262 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:20,263 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:20,263 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:20,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796280058_0001_00_000001, launchTime=1452796280263
2016-01-14 18:31:20,264 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796280058_0001_00_000001
2016-01-14 18:31:20,265 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796280058_0001_1_00_000000_0] to container: [container_1452796280058_0001_00_000001]
2016-01-14 18:31:20,265 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:20,265 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796280058_0001_00_000001 given task: attempt_1452796280058_0001_1_00_000000_0
2016-01-14 18:31:20,265 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796280058_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796280058_0001_1_00_000000_0
2016-01-14 18:31:20 Starting to run new task attempt: attempt_1452796280058_0001_1_00_000000_0
2016-01-14 18:31:20,266 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796280058_0001_1_00_000000_0
2016-01-14 18:31:20,266 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:20,267 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:20,267 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:20,265 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796280058_0001_1_00_000000_0] started. Is using containerId: [container_1452796280058_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:20,267 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:20,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FDCE8BFF601B49C29BAC1835C168CCD0, taskAttemptId=attempt_1452796280058_0001_1_00_000000_0, startTime=1452796280265, containerId=container_1452796280058_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796280058_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:20,268 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:20,268 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:20,268 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280058_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:20,268 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:20,268 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280058_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:20,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: FDCE8BFF601B49C29BAC1835C168CCD0, VertexParallelism: 1, TaskAttemptID:attempt_1452796280058_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2702CC0160CE43DEB62A8E93357E14B8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A11731EDB03E4D6B82B29DFC93C6D189, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:20,269 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:20,270 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796280058_0001_1_00_000000_0
2016-01-14 18:31:20,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:20,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:20,274 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796280058_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:20,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:20,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A11731EDB03E4D6B82B29DFC93C6D189, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:20,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2702CC0160CE43DEB62A8E93357E14B8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:20,277 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:20,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2702CC0160CE43DEB62A8E93357E14B8
2016-01-14 18:31:20,278 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2702CC0160CE43DEB62A8E93357E14B8
2016-01-14 18:31:20,278 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:20,279 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A11731EDB03E4D6B82B29DFC93C6D189
2016-01-14 18:31:20,278 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:20,283 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:20,283 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2702CC0160CE43DEB62A8E93357E14B8
2016-01-14 18:31:20,283 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:20,284 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A11731EDB03E4D6B82B29DFC93C6D189
2016-01-14 18:31:20,283 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2702CC0160CE43DEB62A8E93357E14B8
2016-01-14 18:31:20,284 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A11731EDB03E4D6B82B29DFC93C6D189
2016-01-14 18:31:20,290 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FDCE8BFF601B49C29BAC1835C168CCD0, ordinal: 0
2016-01-14 18:31:20,291 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDCE8BFF601B49C29BAC1835C168CCD0, mem on start (mb), free: 206, total: 490, max: 672
2016-01-14 18:31:20,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:20,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:20,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:20,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:20,291 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:20,294 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:20,294 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:20,295 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:20,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:20,296 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:20,297 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2702CC0160CE43DEB62A8E93357E14B8, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:20,298 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A11731EDB03E4D6B82B29DFC93C6D189, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:20,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:20,299 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2702CC0160CE43DEB62A8E93357E14B8
2016-01-14 18:31:20,299 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:20,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2702CC0160CE43DEB62A8E93357E14B8
2016-01-14 18:31:20,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2702CC0160CE43DEB62A8E93357E14B8 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:20,300 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:20,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:20,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:20,301 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796280058_0001_1_00_000000_0
2016-01-14 18:31:20,303 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:20,305 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:20,306 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2702CC0160CE43DEB62A8E93357E14B8
2016-01-14 18:31:20,306 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 994D049B61F54C74832F2FCA9FEFCE1D
2016-01-14 18:31:20,306 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 994D049B61F54C74832F2FCA9FEFCE1D
2016-01-14 18:31:20,307 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@17ba3d24; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@440412a7; comparator=cascading.tuple.hadoop.util.TupleComparator@6764b4d3; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:20,309 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:20,309 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:20,310 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:20,312 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@cc1ab38
2016-01-14 18:31:20,314 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@cc1ab38
2016-01-14 18:31:20,314 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:20,314 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:20,325 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:20,325 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:20,325 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:20,326 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:20,326 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:20,326 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:20,326 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:20,327 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FDCE8BFF601B49C29BAC1835C168CCD0, all 1 inputs ready in: 00:00:00.001
2016-01-14 18:31:20,330 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796280058_0001_1_00_000000_0
2016-01-14 18:31:20,330 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FDCE8BFF601B49C29BAC1835C168CCD0, mem on close (mb), free: 105, total: 490, max: 672
2016-01-14 18:31:20,330 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:20,330 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:20,331 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:20,331 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:20,333 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:20 Completed running task attempt: attempt_1452796280058_0001_1_00_000000_0
2016-01-14 18:31:20,337 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796280058_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:20,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796280058_0001_1_00, eventCount=1
2016-01-14 18:31:20,338 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FDCE8BFF601B49C29BAC1835C168CCD0, taskAttemptId=attempt_1452796280058_0001_1_00_000000_0, startTime=1452796280265, finishTime=1452796280338, timeTaken=73, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796280327, Process_Duration=3, Process_End_Time=1452796280330, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:20,338 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280058_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:20,338 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:20,338 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:20,338 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796280058_0001_1_00_000000_0
2016-01-14 18:31:20,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:TASK_FINISHED]: vertexName=FDCE8BFF601B49C29BAC1835C168CCD0, taskId=task_1452796280058_0001_1_00_000000, startTime=1452796280265, finishTime=1452796280339, timeTaken=74, status=SUCCEEDED, successfulAttemptID=attempt_1452796280058_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796280327, Process_Duration=3, Process_End_Time=1452796280330, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:20,340 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280058_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:20,340 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0] : 1
2016-01-14 18:31:20,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796280327, Process_Duration=3, Process_End_Time=1452796280330, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:20,340 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:20,340 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:20,340 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:20,340 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796280058_0001_00_000001
2016-01-14 18:31:20,340 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0]
2016-01-14 18:31:20,343 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:VERTEX_FINISHED]: vertexName=FDCE8BFF601B49C29BAC1835C168CCD0, vertexId=vertex_1452796280058_0001_1_00, initRequestedTime=1452796280233, initedTime=1452796280248, startRequestedTime=1452796280249, startedTime=1452796280249, finishTime=1452796280341, timeTaken=92, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796280327, Process_Duration=3, Process_End_Time=1452796280330, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796280265, firstTasksToStart=[ task_1452796280058_0001_1_00_000000 ], lastTaskFinishTime=1452796280338, lastTasksToFinish=[ task_1452796280058_0001_1_00_000000 ], minTaskDuration=73, maxTaskDuration=73, avgTaskDuration=73.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796280058_0001_1_00_000000 ], longestDurationTasks=[ task_1452796280058_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:20,343 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:20,343 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:20,344 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:20,344 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796280058_0001_00_000001, stoppedTime=1452796280344, exitStatus=0
2016-01-14 18:31:20,344 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:20,345 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189] attempt: attempt_1452796280058_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:20,345 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:20,345 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A11731EDB03E4D6B82B29DFC93C6D189
2016-01-14 18:31:20,346 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189]
2016-01-14 18:31:20,346 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:20,346 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796280058_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:20,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:TASK_STARTED]: vertexName=A11731EDB03E4D6B82B29DFC93C6D189, taskId=task_1452796280058_0001_1_01_000000, scheduledTime=1452796280346, launchTime=1452796280346
2016-01-14 18:31:20,346 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280058_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:20,347 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189]
2016-01-14 18:31:20,347 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:20,347 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: A11731EDB03E4D6B82B29DFC93C6D189, VertexParallelism: 1, TaskAttemptID:attempt_1452796280058_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FDCE8BFF601B49C29BAC1835C168CCD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=548ED704E7E346469468760B365DC75B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:20,347 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280058_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:20,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:20,349 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:20,350 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796280058_0001_00_000002
2016-01-14 18:31:20,351 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:20,353 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796280058_0001_00_000002
2016-01-14 18:31:20,353 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:20,353 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796280058_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:20,354 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796280058_0001_00_000001: 13 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:20,354 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796280058_0001_00_000002, launchTime=1452796280354
2016-01-14 18:31:20,354 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:20,354 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:20,355 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796280058_0001_00_000002
2016-01-14 18:31:20,354 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796280058_0001_00_000001, Exiting
2016-01-14 18:31:20,356 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796280058_0001_00_000001
2016-01-14 18:31:20,356 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796280058_0001_00_000001
2016-01-14 18:31:20,356 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796280058_0001_1_01_000000_0] to container: [container_1452796280058_0001_00_000002]
2016-01-14 18:31:20,356 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:20,356 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796280058_0001_00_000002 given task: attempt_1452796280058_0001_1_01_000000_0
2016-01-14 18:31:20,356 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796280058_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796280058_0001_1_01_000000_0
2016-01-14 18:31:20,359 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796280058_0001_1_01_000000_0] started. Is using containerId: [container_1452796280058_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:20,359 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:20,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A11731EDB03E4D6B82B29DFC93C6D189, taskAttemptId=attempt_1452796280058_0001_1_01_000000_0, startTime=1452796280356, containerId=container_1452796280058_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796280058_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:20 Starting to run new task attempt: attempt_1452796280058_0001_1_01_000000_0
2016-01-14 18:31:20,360 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280058_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:20,360 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796280058_0001_1_01_000000_0
2016-01-14 18:31:20,360 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:20,361 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:20,361 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:20,361 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:20,361 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:20,361 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:20,361 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9] every/(1/1) ...lyplatform/-1_UNKNOWN_ALL, VertexName: A11731EDB03E4D6B82B29DFC93C6D189, VertexParallelism: 1, TaskAttemptID:attempt_1452796280058_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FDCE8BFF601B49C29BAC1835C168CCD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=548ED704E7E346469468760B365DC75B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:20,361 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:20,362 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280058_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:20,362 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796280058_0001_1_01_000000_0
2016-01-14 18:31:20,365 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:20,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:20,366 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796280058_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:20,365 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796280058_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:20,367 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796280058_0001_00_000001 completed successfully
2016-01-14 18:31:20,367 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:20,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FDCE8BFF601B49C29BAC1835C168CCD0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:20,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:20,369 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FDCE8BFF601B49C29BAC1835C168CCD0
2016-01-14 18:31:20,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=548ED704E7E346469468760B365DC75B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:20,368 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:20,370 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:20,371 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 548ED704E7E346469468760B365DC75B
2016-01-14 18:31:20,372 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:20,372 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 548ED704E7E346469468760B365DC75B
2016-01-14 18:31:20,373 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:20,374 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:20,375 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:20,375 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FDCE8BFF601B49C29BAC1835C168CCD0
2016-01-14 18:31:20,376 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FDCE8BFF601B49C29BAC1835C168CCD0
2016-01-14 18:31:20,385 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A11731EDB03E4D6B82B29DFC93C6D189, ordinal: 1
2016-01-14 18:31:20,385 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A11731EDB03E4D6B82B29DFC93C6D189, mem on start (mb), free: 98, total: 490, max: 672
2016-01-14 18:31:20,385 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:20,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 548ED704E7E346469468760B365DC75B
2016-01-14 18:31:20,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:20,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:20,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:20,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:20,387 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:20,387 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:20,388 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:20,388 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:20,388 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:20,389 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:20,389 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 548ED704E7E346469468760B365DC75B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:20,389 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FDCE8BFF601B49C29BAC1835C168CCD0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:20,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:20,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FDCE8BFF601B49C29BAC1835C168CCD0 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:20,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FDCE8BFF601B49C29BAC1835C168CCD0
2016-01-14 18:31:20,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:20,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:20,392 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:20,392 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:20,393 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FDCE8BFF601B49C29BAC1835C168CCD0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:20,393 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:20,393 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:20,393 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:20,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FDCE8BFF601B49C29BAC1835C168CCD0
2016-01-14 18:31:20,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:20,395 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796280058_0001_1_01_000000_0
2016-01-14 18:31:20,398 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:20,398 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796280058_0001_1_00_000000_0_10045, runDuration: 55622, ]
2016-01-14 18:31:20,398 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 994D049B61F54C74832F2FCA9FEFCE1D
2016-01-14 18:31:20,400 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL"], id: 548ED704E7E346469468760B365DC75B
2016-01-14 18:31:20,401 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_ALL"]
2016-01-14 18:31:20,401 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 994D049B61F54C74832F2FCA9FEFCE1D, for 1 inputs
2016-01-14 18:31:20,401 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796280058_0001_1_00_000000_0_10045] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796280401, TimeTaken=2, Rate=0.14 MB/s
2016-01-14 18:31:20,402 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FDCE8BFF601B49C29BAC1835C168CCD0
2016-01-14 18:31:20,402 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:20,402 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FDCE8BFF601B49C29BAC1835C168CCD0] #8 in 3ms
2016-01-14 18:31:20,403 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FDCE8BFF601B49C29BAC1835C168CCD0] #8, status:false, isInterrupted:false
2016-01-14 18:31:20,403 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:20,403 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:20,404 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:20,404 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:20,404 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:20,404 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FDCE8BFF601B49C29BAC1835C168CCD0
2016-01-14 18:31:20,404 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:20,404 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A11731EDB03E4D6B82B29DFC93C6D189, all 1 inputs ready in: 00:00:00.003
2016-01-14 18:31:20,406 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@65625078; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@19912a48
2016-01-14 18:31:20,417 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:20,417 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:20,417 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796280058_0001_1_01_000000_0
2016-01-14 18:31:20,417 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796280058_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:20,418 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796280058_0001_1_01_000000_0
2016-01-14 18:31:20,418 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A11731EDB03E4D6B82B29DFC93C6D189, mem on close (mb), free: 94, total: 490, max: 672
2016-01-14 18:31:20,419 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FDCE8BFF601B49C29BAC1835C168CCD0
2016-01-14 18:31:20,419 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:20,419 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:20,419 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:20,419 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:20,420 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:20 Completed running task attempt: attempt_1452796280058_0001_1_01_000000_0
2016-01-14 18:31:20,421 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796280058_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:20,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796280405, Process_Duration=13, Process_End_Time=1452796280418, Read_Duration=8, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:20,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A11731EDB03E4D6B82B29DFC93C6D189, taskAttemptId=attempt_1452796280058_0001_1_01_000000_0, startTime=1452796280356, finishTime=1452796280422, timeTaken=66, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796280405, Process_Duration=13, Process_End_Time=1452796280418, Read_Duration=8, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:20,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:20,423 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280058_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:20,423 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:20,423 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:20,423 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796280058_0001_1_01_000000_0
2016-01-14 18:31:20,423 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796280058_0001_00_000002
2016-01-14 18:31:20,424 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:20,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:TASK_FINISHED]: vertexName=A11731EDB03E4D6B82B29DFC93C6D189, taskId=task_1452796280058_0001_1_01_000000, startTime=1452796280356, finishTime=1452796280423, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1452796280058_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796280405, Process_Duration=13, Process_End_Time=1452796280418, Read_Duration=8, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:20,424 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280058_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:20,425 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189] : 1
2016-01-14 18:31:20,425 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189]
2016-01-14 18:31:20,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:20,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:20,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:VERTEX_FINISHED]: vertexName=A11731EDB03E4D6B82B29DFC93C6D189, vertexId=vertex_1452796280058_0001_1_01, initRequestedTime=1452796280235, initedTime=1452796280242, startRequestedTime=1452796280253, startedTime=1452796280253, finishTime=1452796280425, timeTaken=172, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513802240, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796280405, Process_Duration=13, Process_End_Time=1452796280418, Read_Duration=8, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796280356, firstTasksToStart=[ task_1452796280058_0001_1_01_000000 ], lastTaskFinishTime=1452796280422, lastTasksToFinish=[ task_1452796280058_0001_1_01_000000 ], minTaskDuration=66, maxTaskDuration=66, avgTaskDuration=66.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796280058_0001_1_01_000000 ], longestDurationTasks=[ task_1452796280058_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:20,426 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:20,427 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:20,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796280058_0001_00_000002, stoppedTime=1452796280427, exitStatus=0
2016-01-14 18:31:20,427 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:20,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:20,427 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:20,427 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796280058_0001_1
2016-01-14 18:31:20,428 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796280058_0001_1
2016-01-14 18:31:20,428 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 548ED704E7E346469468760B365DC75B for vertex: vertex_1452796280058_0001_1_01 [A11731EDB03E4D6B82B29DFC93C6D189]
2016-01-14 18:31:20,434 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796280058_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:20,434 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796280058_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:20,434 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796280058_0001_00_000002, Exiting
2016-01-14 18:31:20,435 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796280058_0001_00_000002
2016-01-14 18:31:20,435 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796280058_0001_00_000002
2016-01-14 18:31:20,435 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796280058_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:20,435 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796280058_0001_00_000002 completed successfully
2016-01-14 18:31:20,437 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796280058_0001_1_00 [FDCE8BFF601B49C29BAC1835C168CCD0]
2016-01-14 18:31:20,439 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796280058_0001_1, queueSize=0
2016-01-14 18:31:20,439 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:20,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280058_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796280058_0001_1, startTime=1452796280233, finishTime=1452796280437, timeTaken=204, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1091, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1027604480, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592560732, Process_Duration=16, Process_End_Time=2905592560748, Read_Duration=9, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:20,441 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796280058_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:20,441 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796280058_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:20,441 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280058_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:20,441 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796280058_0001_00_000002
2016-01-14 18:31:20 Completed Dag: dag_1452796280058_0001_1
2016-01-14 18:31:20,441 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796280058_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:20,441 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796280058_0001_1_post
2016-01-14 18:31:20,442 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:20,442 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:20,449 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_UNKNOWN_ALL, applicationId=application_1452796280058_0001
2016-01-14 18:31:20,451 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:20,451 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:20,452 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:20,452 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:20,452 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:20,452 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:20,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:20,453 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:20,453 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:20,453 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:20,453 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:20,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:20,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:20,459 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F5F7A638554C416BBEF207A2716AEAC4/CC741090C23A4112BC4A78C529955EC9/.tez/application_1452796280058_0001
2016-01-14 18:31:20,460 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.422
2016-01-14 18:31:20,538 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_ALL
2016-01-14 18:31:20,565 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:20,566 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:20,580 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:20,580 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:20,614 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 18:31:20,624 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.027
2016-01-14 18:31:20,625 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:20,625 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:20,625 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:20,649 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:20,658 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE
2016-01-14 18:31:20,738 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:20,738 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:20,738 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE"]
2016-01-14 18:31:20,739 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:20,739 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:20,739 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:20,740 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 18:31:20,740 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:20,762 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:20,767 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:20,768 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:20,768 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:20,780 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A/.tez/application_1452796280768_0001 doesn't exist and is created
2016-01-14 18:31:20,800 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:20,800 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:20,801 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A/.tez/application_1452796280768_0001
2016-01-14 18:31:20,804 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796280768_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:20,821 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:20,822 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:20,824 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:20,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:20,824 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A/.tez/application_1452796280768_0001/localmode-log-dir
2016-01-14 18:31:20,825 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A/.tez/application_1452796280768_0001/localmode-log-dir/history.txt.appattempt_1452796280768_0001_000000, maxErrors=10
2016-01-14 18:31:20,825 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:20,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796280768_0001_000000, appSubmitTime=1452796280803, launchTime=1452796280803
2016-01-14 18:31:20,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:20,835 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:20,836 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:20,837 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:20,838 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:35118
2016-01-14 18:31:20,841 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:20,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796280768_0001_000000, startTime=1452796280846
2016-01-14 18:31:20,846 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:20,846 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:20,901 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:20,901 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:20,901 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1452796280768_0001, dagName=[B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 18:31:20,926 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 18:31:20,928 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796280768_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A/.tez/application_1452796280768_0001/localmode-log-dir/dag_1452796280768_0001_1.dot
2016-01-14 18:31:20,929 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A/.tez/application_1452796280768_0001/localmode-log-dir/dag_1452796280768_0001_1-tez-dag.pb.txt
2016-01-14 18:31:20,939 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796280768_0001_1
2016-01-14 18:31:20,939 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:20,939 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 18:31:20 Running Dag: dag_1452796280768_0001_1
2016-01-14 18:31:20,950 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796280768_0001_1, submitTime=1452796280927
2016-01-14 18:31:20,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:20,952 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BA5BEB7372DA4A84AF032F0BDB76C2BB
2016-01-14 18:31:20,952 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:20,953 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:20,954 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796280768_0001_1, initTime=1452796280950
2016-01-14 18:31:20,954 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796280768_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:20,955 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:DAG_STARTED]: dagID=dag_1452796280768_0001_1, startTime=1452796280954
2016-01-14 18:31:20,955 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:20,955 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796280768_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:20,955 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A41B7F74D3E04AB3957BE115A049D729 : {C4D55412FD1E49F4B8990CC72DBA8693={InputName=C4D55412FD1E49F4B8990CC72DBA8693}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:20,955 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C4D55412FD1E49F4B8990CC72DBA8693, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:20,955 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729]
2016-01-14 18:31:20,956 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1452796280768_0001, dagName=[B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A] every/(1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 18:31:20,956 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729]
2016-01-14 18:31:20,956 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729]
2016-01-14 18:31:20,956 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729]. Starting root input initializers: 1
2016-01-14 18:31:20,955 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:20,957 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:20,957 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB]
2016-01-14 18:31:20,957 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C4D55412FD1E49F4B8990CC72DBA8693 on vertex vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729]
2016-01-14 18:31:20,959 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:20,959 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C4D55412FD1E49F4B8990CC72DBA8693 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:20,961 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:20,961 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:20,961 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB]
2016-01-14 18:31:20,961 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:20,961 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB]
2016-01-14 18:31:20,962 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB]
2016-01-14 18:31:20,962 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B0B527DC75ED4B1C81EED18C8283EC5F, vertexId=vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:20,962 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B0B527DC75ED4B1C81EED18C8283EC5F, vertexId=vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB]
2016-01-14 18:31:20,963 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:20,964 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C4D55412FD1E49F4B8990CC72DBA8693 on vertex vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729]
2016-01-14 18:31:20,964 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B0B527DC75ED4B1C81EED18C8283EC5F, vertexName=BA5BEB7372DA4A84AF032F0BDB76C2BB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:20,965 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B0B527DC75ED4B1C81EED18C8283EC5F, vertexId=vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB]
2016-01-14 18:31:20,965 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:20,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BA5BEB7372DA4A84AF032F0BDB76C2BB, vertexId=vertex_1452796280768_0001_1_01, initRequestedTime=1452796280957, initedTime=1452796280965, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:20,966 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:20,966 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:20,966 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C4D55412FD1E49F4B8990CC72DBA8693=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:20,967 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729] parallelism set to 1
2016-01-14 18:31:20,967 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:20,967 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796280768_0001_1_00, eventCount=1
2016-01-14 18:31:20,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:20,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A41B7F74D3E04AB3957BE115A049D729, vertexId=vertex_1452796280768_0001_1_00, initRequestedTime=1452796280955, initedTime=1452796280968, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:20,968 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729] with distanceFromRoot: 0
2016-01-14 18:31:20,968 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:20,969 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A41B7F74D3E04AB3957BE115A049D729
2016-01-14 18:31:20,969 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729]
2016-01-14 18:31:20,969 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:20,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796280768_0001_1_00, startRequestedTime=1452796280969, startedTime=1452796280969
2016-01-14 18:31:20,970 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:20,970 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:20,971 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:TASK_STARTED]: vertexName=A41B7F74D3E04AB3957BE115A049D729, taskId=task_1452796280768_0001_1_00_000000, scheduledTime=1452796280970, launchTime=1452796280970
2016-01-14 18:31:20,971 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280768_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:20,971 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796280768_0001_1_00 for vertex: vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] numStartedSources: 1 numSources: 1
2016-01-14 18:31:20,971 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] with distanceFromRoot: 1
2016-01-14 18:31:20,971 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729]
2016-01-14 18:31:20,972 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A41B7F74D3E04AB3957BE115A049D729 in vertex: BA5BEB7372DA4A84AF032F0BDB76C2BB
2016-01-14 18:31:20,972 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BA5BEB7372DA4A84AF032F0BDB76C2BB with 1 source tasks and 1 pending tasks
2016-01-14 18:31:20,972 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:20,973 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796280768_0001_1_01, startRequestedTime=1452796280971, startedTime=1452796280972
2016-01-14 18:31:20,973 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:20,973 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: A41B7F74D3E04AB3957BE115A049D729, VertexParallelism: 1, TaskAttemptID:attempt_1452796280768_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C4D55412FD1E49F4B8990CC72DBA8693, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BA5BEB7372DA4A84AF032F0BDB76C2BB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:20,974 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280768_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:20,974 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:20,976 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:20,976 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:20,977 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:20,977 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796280768_0001_00_000001
2016-01-14 18:31:20,977 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280768_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:20,980 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796280768_0001_00_000001
2016-01-14 18:31:20,980 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:20,981 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280768_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:20,981 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:20,981 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796280768_0001_00_000001, launchTime=1452796280981
2016-01-14 18:31:20,985 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796280768_0001, with dag id: dag_1452796280768_0001_1
2016-01-14 18:31:20,986 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796280768_0001_00_000001
2016-01-14 18:31:20,987 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796280768_0001_1_00_000000_0] to container: [container_1452796280768_0001_00_000001]
2016-01-14 18:31:20,988 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280768_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:20,988 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796280768_0001_00_000001 given task: attempt_1452796280768_0001_1_00_000000_0
2016-01-14 18:31:20,989 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796280768_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796280768_0001_1_00_000000_0
2016-01-14 18:31:20,989 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796280768_0001_1_00_000000_0] started. Is using containerId: [container_1452796280768_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:20,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A41B7F74D3E04AB3957BE115A049D729, taskAttemptId=attempt_1452796280768_0001_1_00_000000_0, startTime=1452796280988, containerId=container_1452796280768_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796280768_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:20,989 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:20,989 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280768_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:20,990 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280768_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:20 Starting to run new task attempt: attempt_1452796280768_0001_1_00_000000_0
2016-01-14 18:31:20,992 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796280768_0001_1_00_000000_0
2016-01-14 18:31:20,992 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:20,994 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:20,994 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:20,995 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:20,995 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:20,996 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:20,996 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: A41B7F74D3E04AB3957BE115A049D729, VertexParallelism: 1, TaskAttemptID:attempt_1452796280768_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C4D55412FD1E49F4B8990CC72DBA8693, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BA5BEB7372DA4A84AF032F0BDB76C2BB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:20,997 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:20,997 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796280768_0001_1_00_000000_0
2016-01-14 18:31:20,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:20,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:20,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C4D55412FD1E49F4B8990CC72DBA8693, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:20,999 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BA5BEB7372DA4A84AF032F0BDB76C2BB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:20,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:21,000 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796280768_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:21,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:21,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C4D55412FD1E49F4B8990CC72DBA8693
2016-01-14 18:31:21,001 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C4D55412FD1E49F4B8990CC72DBA8693
2016-01-14 18:31:21,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:21,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BA5BEB7372DA4A84AF032F0BDB76C2BB
2016-01-14 18:31:21,003 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:21,005 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:21,005 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BA5BEB7372DA4A84AF032F0BDB76C2BB
2016-01-14 18:31:21,006 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BA5BEB7372DA4A84AF032F0BDB76C2BB
2016-01-14 18:31:21,008 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:21,008 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C4D55412FD1E49F4B8990CC72DBA8693
2016-01-14 18:31:21,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C4D55412FD1E49F4B8990CC72DBA8693
2016-01-14 18:31:21,011 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A41B7F74D3E04AB3957BE115A049D729, ordinal: 0
2016-01-14 18:31:21,011 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A41B7F74D3E04AB3957BE115A049D729, mem on start (mb), free: 198, total: 490, max: 672
2016-01-14 18:31:21,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:21,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:21,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:21,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:21,012 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:21,012 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:21,012 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:21,013 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:21,013 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:21,013 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:21,013 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C4D55412FD1E49F4B8990CC72DBA8693, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:21,013 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BA5BEB7372DA4A84AF032F0BDB76C2BB, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:21,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:21,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C4D55412FD1E49F4B8990CC72DBA8693 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:21,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:21,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:21,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C4D55412FD1E49F4B8990CC72DBA8693
2016-01-14 18:31:21,014 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:21,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C4D55412FD1E49F4B8990CC72DBA8693
2016-01-14 18:31:21,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:21,015 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796280768_0001_1_00_000000_0
2016-01-14 18:31:21,015 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:21,016 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@23c54e1a
2016-01-14 18:31:21,017 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@23c54e1a
2016-01-14 18:31:21,017 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:21,017 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:21,019 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:21,019 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C4D55412FD1E49F4B8990CC72DBA8693
2016-01-14 18:31:21,020 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: AEE3602952604133996AD3C851D9EFFC
2016-01-14 18:31:21,020 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] AEE3602952604133996AD3C851D9EFFC
2016-01-14 18:31:21,020 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@cf4bd0f; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@17a82a6e; comparator=cascading.tuple.hadoop.util.TupleComparator@2179a7e4; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:21,021 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:21,021 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:21,021 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:21,034 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:21,035 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:21,035 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:21,035 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:21,035 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:21,036 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:21,036 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:21,036 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A41B7F74D3E04AB3957BE115A049D729, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:21,039 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796280768_0001_1_00_000000_0
2016-01-14 18:31:21,039 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A41B7F74D3E04AB3957BE115A049D729, mem on close (mb), free: 96, total: 490, max: 672
2016-01-14 18:31:21,039 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:21,039 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:21,040 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:21,040 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:21,042 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:21 Completed running task attempt: attempt_1452796280768_0001_1_00_000000_0
2016-01-14 18:31:21,045 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796280768_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:21,046 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796281036, Process_Duration=3, Process_End_Time=1452796281039, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:21,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796280768_0001_1_00, eventCount=1
2016-01-14 18:31:21,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A41B7F74D3E04AB3957BE115A049D729, taskAttemptId=attempt_1452796280768_0001_1_00_000000_0, startTime=1452796280988, finishTime=1452796281046, timeTaken=58, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796281036, Process_Duration=3, Process_End_Time=1452796281039, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:21,047 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280768_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:21,047 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:21,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:21,047 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796280768_0001_1_00_000000_0
2016-01-14 18:31:21,048 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:21,048 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:21,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:21,049 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796280768_0001_00_000001
2016-01-14 18:31:21,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:TASK_FINISHED]: vertexName=A41B7F74D3E04AB3957BE115A049D729, taskId=task_1452796280768_0001_1_00_000000, startTime=1452796280988, finishTime=1452796281048, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1452796280768_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796281036, Process_Duration=3, Process_End_Time=1452796281039, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:21,049 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280768_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:21,049 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729] : 1
2016-01-14 18:31:21,050 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729]
2016-01-14 18:31:21,051 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:VERTEX_FINISHED]: vertexName=A41B7F74D3E04AB3957BE115A049D729, vertexId=vertex_1452796280768_0001_1_00, initRequestedTime=1452796280955, initedTime=1452796280968, startRequestedTime=1452796280969, startedTime=1452796280969, finishTime=1452796281050, timeTaken=81, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796281036, Process_Duration=3, Process_End_Time=1452796281039, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796280988, firstTasksToStart=[ task_1452796280768_0001_1_00_000000 ], lastTaskFinishTime=1452796281046, lastTasksToFinish=[ task_1452796280768_0001_1_00_000000 ], minTaskDuration=58, maxTaskDuration=58, avgTaskDuration=58.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796280768_0001_1_00_000000 ], longestDurationTasks=[ task_1452796280768_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:21,051 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:21,051 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280768_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:21,051 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796280768_0001_00_000001, stoppedTime=1452796281051, exitStatus=0
2016-01-14 18:31:21,051 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280768_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:21,052 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] attempt: attempt_1452796280768_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:21,052 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:21,052 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BA5BEB7372DA4A84AF032F0BDB76C2BB
2016-01-14 18:31:21,052 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB]
2016-01-14 18:31:21,052 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:21,052 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796280768_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:21,052 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:TASK_STARTED]: vertexName=BA5BEB7372DA4A84AF032F0BDB76C2BB, taskId=task_1452796280768_0001_1_01_000000, scheduledTime=1452796281052, launchTime=1452796281052
2016-01-14 18:31:21,053 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:21,053 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280768_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:21,053 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:21,053 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB]
2016-01-14 18:31:21,053 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280768_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:21,053 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: BA5BEB7372DA4A84AF032F0BDB76C2BB, VertexParallelism: 1, TaskAttemptID:attempt_1452796280768_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A41B7F74D3E04AB3957BE115A049D729, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B0B527DC75ED4B1C81EED18C8283EC5F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:21,053 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280768_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:21,054 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:21,054 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796280768_0001_00_000002
2016-01-14 18:31:21,054 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280768_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:21,056 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796280768_0001_00_000002
2016-01-14 18:31:21,056 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:21,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796280768_0001_00_000002, launchTime=1452796281057
2016-01-14 18:31:21,058 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280768_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:21,058 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:21,058 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796280768_0001_00_000002
2016-01-14 18:31:21,059 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796280768_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:21,059 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796280768_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:21,059 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796280768_0001_00_000001, Exiting
2016-01-14 18:31:21,059 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796280768_0001_00_000001
2016-01-14 18:31:21,059 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796280768_0001_00_000001
2016-01-14 18:31:21,060 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796280768_0001_1_01_000000_0] to container: [container_1452796280768_0001_00_000002]
2016-01-14 18:31:21,061 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280768_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:21,061 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796280768_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:21,061 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796280768_0001_1_01_000000_0] started. Is using containerId: [container_1452796280768_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:21,061 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796280768_0001_00_000001 completed successfully
2016-01-14 18:31:21,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BA5BEB7372DA4A84AF032F0BDB76C2BB, taskAttemptId=attempt_1452796280768_0001_1_01_000000_0, startTime=1452796281061, containerId=container_1452796280768_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796280768_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:21,062 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796280768_0001_00_000002 given task: attempt_1452796280768_0001_1_01_000000_0
2016-01-14 18:31:21,062 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280768_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:21,062 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796280768_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796280768_0001_1_01_000000_0
2016-01-14 18:31:21,062 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280768_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:21,062 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:21 Starting to run new task attempt: attempt_1452796280768_0001_1_01_000000_0
2016-01-14 18:31:21,063 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796280768_0001_1_01_000000_0
2016-01-14 18:31:21,063 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:21,063 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:21,063 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:21,063 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:21,063 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:21,063 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:21,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A] every/(1/1) ...atform/-1_UNKNOWN_REPLACE, VertexName: BA5BEB7372DA4A84AF032F0BDB76C2BB, VertexParallelism: 1, TaskAttemptID:attempt_1452796280768_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A41B7F74D3E04AB3957BE115A049D729, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B0B527DC75ED4B1C81EED18C8283EC5F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:21,063 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280768_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:21,064 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:21,064 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796280768_0001_1_01_000000_0
2016-01-14 18:31:21,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:21,064 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:21,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A41B7F74D3E04AB3957BE115A049D729, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:21,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:21,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A41B7F74D3E04AB3957BE115A049D729
2016-01-14 18:31:21,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B0B527DC75ED4B1C81EED18C8283EC5F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:21,066 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796280768_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:21,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:21,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:21,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B0B527DC75ED4B1C81EED18C8283EC5F
2016-01-14 18:31:21,068 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:21,068 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B0B527DC75ED4B1C81EED18C8283EC5F
2016-01-14 18:31:21,072 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:21,080 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:21,080 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:21,080 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BA5BEB7372DA4A84AF032F0BDB76C2BB, ordinal: 1
2016-01-14 18:31:21,081 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA5BEB7372DA4A84AF032F0BDB76C2BB, mem on start (mb), free: 90, total: 490, max: 672
2016-01-14 18:31:21,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:21,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:21,081 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:21,082 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A41B7F74D3E04AB3957BE115A049D729
2016-01-14 18:31:21,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A41B7F74D3E04AB3957BE115A049D729
2016-01-14 18:31:21,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B0B527DC75ED4B1C81EED18C8283EC5F
2016-01-14 18:31:21,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:21,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:21,084 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:21,084 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:21,085 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:21,085 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:21,085 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:21,086 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:21,086 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B0B527DC75ED4B1C81EED18C8283EC5F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:21,086 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A41B7F74D3E04AB3957BE115A049D729, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:21,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:21,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A41B7F74D3E04AB3957BE115A049D729 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:21,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A41B7F74D3E04AB3957BE115A049D729
2016-01-14 18:31:21,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:21,088 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:21,088 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:21,089 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:21,089 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A41B7F74D3E04AB3957BE115A049D729 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:21,090 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:21,090 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:21,090 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:21,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A41B7F74D3E04AB3957BE115A049D729
2016-01-14 18:31:21,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:21,093 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796280768_0001_1_01_000000_0
2016-01-14 18:31:21,093 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796280768_0001_1_00_000000_0_10051, runDuration: 41402, ]
2016-01-14 18:31:21,095 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796280768_0001_1_00_000000_0_10051] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796281095, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:21,096 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A41B7F74D3E04AB3957BE115A049D729
2016-01-14 18:31:21,096 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:21,096 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A41B7F74D3E04AB3957BE115A049D729] #9 in 2ms
2016-01-14 18:31:21,097 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A41B7F74D3E04AB3957BE115A049D729] #9, status:false, isInterrupted:false
2016-01-14 18:31:21,097 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:21,097 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:21,097 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:21,098 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:21,098 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:21,098 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A41B7F74D3E04AB3957BE115A049D729
2016-01-14 18:31:21,098 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:21,098 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:21,099 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: AEE3602952604133996AD3C851D9EFFC
2016-01-14 18:31:21,100 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE"], id: B0B527DC75ED4B1C81EED18C8283EC5F
2016-01-14 18:31:21,100 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_REPLACE"]
2016-01-14 18:31:21,100 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] AEE3602952604133996AD3C851D9EFFC, for 1 inputs
2016-01-14 18:31:21,101 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BA5BEB7372DA4A84AF032F0BDB76C2BB, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:21,102 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@285a7e4b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@4d19829f
2016-01-14 18:31:21,104 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:21,111 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:21,116 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:21,116 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:21,117 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796280768_0001_1_01_000000_0
2016-01-14 18:31:21,117 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796280768_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:21,118 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796280768_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:21,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796281101, Process_Duration=17, Process_End_Time=1452796281118, Read_Duration=1, Tuples_Read=1]]
2016-01-14 18:31:21,128 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BA5BEB7372DA4A84AF032F0BDB76C2BB, mem on close (mb), free: 85, total: 490, max: 672
2016-01-14 18:31:21,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=BA5BEB7372DA4A84AF032F0BDB76C2BB, index=1
2016-01-14 18:31:21,129 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A41B7F74D3E04AB3957BE115A049D729
2016-01-14 18:31:21,129 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:21,130 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:21,130 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:21,130 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:21,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BA5BEB7372DA4A84AF032F0BDB76C2BB, taskAttemptId=attempt_1452796280768_0001_1_01_000000_0, startTime=1452796281061, finishTime=1452796281128, timeTaken=67, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=2, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796281101, Process_Duration=17, Process_End_Time=1452796281118, Read_Duration=1, Tuples_Read=1
2016-01-14 18:31:21,131 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:21,131 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280768_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:21,132 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:21,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=BA5BEB7372DA4A84AF032F0BDB76C2BB, sourceVertex=A41B7F74D3E04AB3957BE115A049D729
2016-01-14 18:31:21,134 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=BA5BEB7372DA4A84AF032F0BDB76C2BB, sourceVertex=B0B527DC75ED4B1C81EED18C8283EC5F
2016-01-14 18:31:21,133 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796280768_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:21,134 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796280768_0001_00_000002
2016-01-14 18:31:21,134 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796280768_0001_00_000002
2016-01-14 18:31:21,134 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:21,135 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796280768_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:21,135 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796280768_0001_00_000002 completed but with errors
2016-01-14 18:31:21,135 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:TASK_FINISHED]: vertexName=BA5BEB7372DA4A84AF032F0BDB76C2BB, taskId=task_1452796280768_0001_1_01_000000, startTime=1452796281061, finishTime=1452796281134, timeTaken=73, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:21,135 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796280768_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:21,137 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:21,138 INFO container.AMContainerImpl (AMContainerImpl.java:transition(686)) - Container container_1452796280768_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:21,138 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:21,138 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796280768_0001_00_000002, stoppedTime=1452796281138, exitStatus=1
2016-01-14 18:31:21,139 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796280768_0001_00_000002 transitioned from RUNNING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:21,139 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] : 1
2016-01-14 18:31:21,139 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] because task failed: task_1452796280768_0001_1_01_000000
2016-01-14 18:31:21,140 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:21,141 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:21,141 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB]
2016-01-14 18:31:21,142 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=B0B527DC75ED4B1C81EED18C8283EC5F, vertexId=vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB]
2016-01-14 18:31:21,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:21,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:VERTEX_FINISHED]: vertexName=BA5BEB7372DA4A84AF032F0BDB76C2BB, vertexId=vertex_1452796280768_0001_1_01, initRequestedTime=1452796280957, initedTime=1452796280965, startRequestedTime=1452796280971, startedTime=1452796280972, finishTime=1452796281145, timeTaken=173, status=FAILED, diagnostics=Task failed, taskId=task_1452796280768_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796281061, firstTasksToStart=[ task_1452796280768_0001_1_01_000000 ], lastTaskFinishTime=1452796281128, lastTasksToFinish=[ task_1452796280768_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:21,147 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:21,149 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796280768_0001_1_01_000000_0 failed: true container: container_1452796280768_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:21,151 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796280768_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:21,152 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:21,153 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:21,153 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796280768_0001_1
2016-01-14 18:31:21,153 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: B0B527DC75ED4B1C81EED18C8283EC5F for vertex: vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB]
2016-01-14 18:31:21,154 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796280768_0001_1_00 [A41B7F74D3E04AB3957BE115A049D729]
2016-01-14 18:31:21,156 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796280768_0001_1, queueSize=0
2016-01-14 18:31:21,156 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:21,156 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796280768_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796280768_0001_1, startTime=1452796280954, finishTime=1452796281154, timeTaken=200, status=FAILED, diagnostics=Vertex failed, vertexName=BA5BEB7372DA4A84AF032F0BDB76C2BB, vertexId=vertex_1452796280768_0001_1_01, diagnostics=[Task failed, taskId=task_1452796280768_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796280768_0001_1_01 [BA5BEB7372DA4A84AF032F0BDB76C2BB] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=514326528, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796281036, Process_Duration=3, Process_End_Time=1452796281039, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:21,156 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796280768_0001_1 finished with state: FAILED
2016-01-14 18:31:21,156 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796280768_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:21 Completed Dag: dag_1452796280768_0001_1
2016-01-14 18:31:21,157 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796280768_0001_1, dagState=FAILED
2016-01-14 18:31:21,157 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796280768_0001_1_post
2016-01-14 18:31:21,157 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:21,157 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:21,166 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/-1_UNKNOWN_REPLACE, applicationId=application_1452796280768_0001
2016-01-14 18:31:21,170 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:21,170 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:21,170 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:21,170 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:21,170 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:21,171 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:21,171 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:21,172 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:21,172 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:21,172 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:21,172 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:21,172 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:21,173 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:21,177 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/B147218280CA4FADA2F4F402EA5AC61B/A58D1CB378F248468FC55070627D1B3A/.tez/application_1452796280768_0001
2016-01-14 18:31:21,177 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:21,177 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...atform/-1_UNKNOWN_REPLACE
2016-01-14 18:31:21,178 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:21,179 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.440
2016-01-14 18:31:21,179 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_REPLACE
2016-01-14 18:31:21,205 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:21,205 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:21,246 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 18:31:21,265 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.031
2016-01-14 18:31:21,265 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:21,266 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:21,266 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:21,292 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:21,302 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP
2016-01-14 18:31:21,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:21,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:21,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP"]
2016-01-14 18:31:21,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:21,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:21,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:21,399 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-01-14 18:31:21,399 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:21,421 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:21,427 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:21,427 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:21,427 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:21,438 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409/.tez/application_1452796281427_0001 doesn't exist and is created
2016-01-14 18:31:21,452 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:21,452 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:21,453 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409/.tez/application_1452796281427_0001
2016-01-14 18:31:21,455 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796281427_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:21,466 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:21,467 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:21,469 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:21,469 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:21,469 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409/.tez/application_1452796281427_0001/localmode-log-dir
2016-01-14 18:31:21,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409/.tez/application_1452796281427_0001/localmode-log-dir/history.txt.appattempt_1452796281427_0001_000000, maxErrors=10
2016-01-14 18:31:21,470 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:21,470 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796281427_0001_000000, appSubmitTime=1452796281454, launchTime=1452796281454
2016-01-14 18:31:21,473 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:21,477 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:21,478 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:53222
2016-01-14 18:31:21,478 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:21,479 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:21,480 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:21,482 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796281427_0001_000000, startTime=1452796281482
2016-01-14 18:31:21,482 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:21,482 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:21,503 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:21,503 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:21,552 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:21,552 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:21,552 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1452796281427_0001, dagName=[04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-01-14 18:31:21,573 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-01-14 18:31:21,574 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796281427_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409/.tez/application_1452796281427_0001/localmode-log-dir/dag_1452796281427_0001_1.dot
2016-01-14 18:31:21,575 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409/.tez/application_1452796281427_0001/localmode-log-dir/dag_1452796281427_0001_1-tez-dag.pb.txt
2016-01-14 18:31:21,586 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796281427_0001_1
2016-01-14 18:31:21,587 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:21,587 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-01-14 18:31:21 Running Dag: dag_1452796281427_0001_1
2016-01-14 18:31:21,598 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:21,599 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796281427_0001_1, submitTime=1452796281573
2016-01-14 18:31:21,599 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F13F4E287F4F488E936F141D71351050
2016-01-14 18:31:21,600 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:21,600 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:21,600 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796281427_0001_1, initTime=1452796281599
2016-01-14 18:31:21,600 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796281427_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:21,601 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:DAG_STARTED]: dagID=dag_1452796281427_0001_1, startTime=1452796281601
2016-01-14 18:31:21,601 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:21,601 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:21,602 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796281427_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:21,602 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A70A8C218720443BABD1638138CF317E : {E43EE0D36F554542BC0C68C8B42E66D0={InputName=E43EE0D36F554542BC0C68C8B42E66D0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:21,602 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E43EE0D36F554542BC0C68C8B42E66D0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:21,602 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E]
2016-01-14 18:31:21,603 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E]
2016-01-14 18:31:21,603 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E]
2016-01-14 18:31:21,603 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E]. Starting root input initializers: 1
2016-01-14 18:31:21,604 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1452796281427_0001, dagName=[04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP
2016-01-14 18:31:21,604 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:21,604 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050]
2016-01-14 18:31:21,604 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E43EE0D36F554542BC0C68C8B42E66D0 on vertex vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E]
2016-01-14 18:31:21,607 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:21,607 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:21,608 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050]
2016-01-14 18:31:21,609 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050]
2016-01-14 18:31:21,609 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050]
2016-01-14 18:31:21,609 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C20AA770125847EA8B143E49466936BC, vertexId=vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:21,609 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C20AA770125847EA8B143E49466936BC, vertexId=vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050]
2016-01-14 18:31:21,611 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E43EE0D36F554542BC0C68C8B42E66D0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:21,612 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:21,612 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:21,613 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C20AA770125847EA8B143E49466936BC, vertexName=F13F4E287F4F488E936F141D71351050, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:21,613 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C20AA770125847EA8B143E49466936BC, vertexId=vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050]
2016-01-14 18:31:21,614 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:21,614 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F13F4E287F4F488E936F141D71351050, vertexId=vertex_1452796281427_0001_1_01, initRequestedTime=1452796281604, initedTime=1452796281614, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:21,615 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:21,619 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:21,619 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E43EE0D36F554542BC0C68C8B42E66D0 on vertex vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E]
2016-01-14 18:31:21,620 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:21,620 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E43EE0D36F554542BC0C68C8B42E66D0=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:21,620 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E] parallelism set to 1
2016-01-14 18:31:21,620 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:21,623 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796281427_0001_1_00, eventCount=1
2016-01-14 18:31:21,624 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A70A8C218720443BABD1638138CF317E, vertexId=vertex_1452796281427_0001_1_00, initRequestedTime=1452796281602, initedTime=1452796281623, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:21,624 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:21,625 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E] with distanceFromRoot: 0
2016-01-14 18:31:21,625 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:21,627 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A70A8C218720443BABD1638138CF317E
2016-01-14 18:31:21,627 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E]
2016-01-14 18:31:21,627 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:21,628 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796281427_0001_1_00, startRequestedTime=1452796281627, startedTime=1452796281627
2016-01-14 18:31:21,628 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:21,629 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796281427_0001, with dag id: dag_1452796281427_0001_1
2016-01-14 18:31:21,629 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:21,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:TASK_STARTED]: vertexName=A70A8C218720443BABD1638138CF317E, taskId=task_1452796281427_0001_1_00_000000, scheduledTime=1452796281628, launchTime=1452796281628
2016-01-14 18:31:21,630 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796281427_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:21,630 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796281427_0001_1_00 for vertex: vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050] numStartedSources: 1 numSources: 1
2016-01-14 18:31:21,630 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050] with distanceFromRoot: 1
2016-01-14 18:31:21,630 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E]
2016-01-14 18:31:21,631 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A70A8C218720443BABD1638138CF317E in vertex: F13F4E287F4F488E936F141D71351050
2016-01-14 18:31:21,631 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F13F4E287F4F488E936F141D71351050 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:21,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796281427_0001_1_01, startRequestedTime=1452796281631, startedTime=1452796281631
2016-01-14 18:31:21,631 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:21,631 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:21,631 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: A70A8C218720443BABD1638138CF317E, VertexParallelism: 1, TaskAttemptID:attempt_1452796281427_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E43EE0D36F554542BC0C68C8B42E66D0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F13F4E287F4F488E936F141D71351050, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:21,633 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796281427_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:21,633 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:21,634 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:21,634 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:21,635 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:21,636 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796281427_0001_00_000001
2016-01-14 18:31:21,636 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:21,638 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796281427_0001_00_000001
2016-01-14 18:31:21,638 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:21,639 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:21,640 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:21,640 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796281427_0001_00_000001, launchTime=1452796281639
2016-01-14 18:31:21,642 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796281427_0001_00_000001
2016-01-14 18:31:21,642 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796281427_0001_1_00_000000_0] to container: [container_1452796281427_0001_00_000001]
2016-01-14 18:31:21,642 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:21,643 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796281427_0001_1_00_000000_0] started. Is using containerId: [container_1452796281427_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:21,643 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A70A8C218720443BABD1638138CF317E, taskAttemptId=attempt_1452796281427_0001_1_00_000000_0, startTime=1452796281642, containerId=container_1452796281427_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796281427_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:21,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:21,643 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796281427_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:21,643 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796281427_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:21,643 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796281427_0001_00_000001 given task: attempt_1452796281427_0001_1_00_000000_0
2016-01-14 18:31:21,643 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796281427_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796281427_0001_1_00_000000_0
2016-01-14 18:31:21 Starting to run new task attempt: attempt_1452796281427_0001_1_00_000000_0
2016-01-14 18:31:21,644 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796281427_0001_1_00_000000_0
2016-01-14 18:31:21,644 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:21,644 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:21,644 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:21,644 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:21,645 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:21,645 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:21,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: A70A8C218720443BABD1638138CF317E, VertexParallelism: 1, TaskAttemptID:attempt_1452796281427_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E43EE0D36F554542BC0C68C8B42E66D0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F13F4E287F4F488E936F141D71351050, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:21,645 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:21,645 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796281427_0001_1_00_000000_0
2016-01-14 18:31:21,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:21,646 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:21,646 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796281427_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:21,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E43EE0D36F554542BC0C68C8B42E66D0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:21,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:21,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:21,647 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F13F4E287F4F488E936F141D71351050, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:21,649 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:21,649 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F13F4E287F4F488E936F141D71351050
2016-01-14 18:31:21,651 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:21,651 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F13F4E287F4F488E936F141D71351050
2016-01-14 18:31:21,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F13F4E287F4F488E936F141D71351050
2016-01-14 18:31:21,651 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:21,652 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E43EE0D36F554542BC0C68C8B42E66D0
2016-01-14 18:31:21,653 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E43EE0D36F554542BC0C68C8B42E66D0
2016-01-14 18:31:21,658 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:21,658 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E43EE0D36F554542BC0C68C8B42E66D0
2016-01-14 18:31:21,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E43EE0D36F554542BC0C68C8B42E66D0
2016-01-14 18:31:21,663 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A70A8C218720443BABD1638138CF317E, ordinal: 0
2016-01-14 18:31:21,663 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A70A8C218720443BABD1638138CF317E, mem on start (mb), free: 203, total: 489, max: 672
2016-01-14 18:31:21,664 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:21,664 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:21,664 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:21,665 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:21,665 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:21,666 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:21,666 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:21,667 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:21,668 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:21,668 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:21,669 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F13F4E287F4F488E936F141D71351050, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:21,669 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E43EE0D36F554542BC0C68C8B42E66D0, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:21,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:21,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E43EE0D36F554542BC0C68C8B42E66D0 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:21,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E43EE0D36F554542BC0C68C8B42E66D0
2016-01-14 18:31:21,671 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:21,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E43EE0D36F554542BC0C68C8B42E66D0
2016-01-14 18:31:21,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:21,672 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:21,672 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:21,672 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796281427_0001_1_00_000000_0
2016-01-14 18:31:21,673 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:21,676 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3777cf76
2016-01-14 18:31:21,677 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3777cf76
2016-01-14 18:31:21,677 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:21,678 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:21,677 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:21,678 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E43EE0D36F554542BC0C68C8B42E66D0
2016-01-14 18:31:21,678 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 57DEDEE8A3CD4EB6818776917B7BCE93
2016-01-14 18:31:21,679 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 57DEDEE8A3CD4EB6818776917B7BCE93
2016-01-14 18:31:21,679 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1d60bd25; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@3a21d734; comparator=cascading.tuple.hadoop.util.TupleComparator@3b7b4bd; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:21,680 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:21,680 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:21,680 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:21,693 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:21,693 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:21,693 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:21,694 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:21,694 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:21,694 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:21,695 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:21,695 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A70A8C218720443BABD1638138CF317E, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:21,698 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796281427_0001_1_00_000000_0
2016-01-14 18:31:21,699 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A70A8C218720443BABD1638138CF317E, mem on close (mb), free: 101, total: 489, max: 672
2016-01-14 18:31:21,699 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:21,699 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:21,699 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:21,699 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:21,702 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:21 Completed running task attempt: attempt_1452796281427_0001_1_00_000000_0
2016-01-14 18:31:21,706 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796281427_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:21,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796281427_0001_1_00, eventCount=1
2016-01-14 18:31:21,708 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513277952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796281695, Process_Duration=3, Process_End_Time=1452796281698, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:21,708 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A70A8C218720443BABD1638138CF317E, taskAttemptId=attempt_1452796281427_0001_1_00_000000_0, startTime=1452796281642, finishTime=1452796281707, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513277952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796281695, Process_Duration=3, Process_End_Time=1452796281698, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:21,708 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:21,708 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796281427_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:21,709 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:21,708 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:21,709 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:21,709 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796281427_0001_1_00_000000_0
2016-01-14 18:31:21,710 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:21,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:TASK_FINISHED]: vertexName=A70A8C218720443BABD1638138CF317E, taskId=task_1452796281427_0001_1_00_000000, startTime=1452796281642, finishTime=1452796281709, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1452796281427_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513277952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796281695, Process_Duration=3, Process_End_Time=1452796281698, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:21,710 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796281427_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:21,710 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E] : 1
2016-01-14 18:31:21,710 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E]
2016-01-14 18:31:21,711 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796281427_0001_00_000001
2016-01-14 18:31:21,712 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:21,712 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:VERTEX_FINISHED]: vertexName=A70A8C218720443BABD1638138CF317E, vertexId=vertex_1452796281427_0001_1_00, initRequestedTime=1452796281602, initedTime=1452796281623, startRequestedTime=1452796281627, startedTime=1452796281627, finishTime=1452796281710, timeTaken=83, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513277952, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796281695, Process_Duration=3, Process_End_Time=1452796281698, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796281642, firstTasksToStart=[ task_1452796281427_0001_1_00_000000 ], lastTaskFinishTime=1452796281707, lastTasksToFinish=[ task_1452796281427_0001_1_00_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796281427_0001_1_00_000000 ], longestDurationTasks=[ task_1452796281427_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:21,713 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:21,713 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:21,713 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050] attempt: attempt_1452796281427_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:21,714 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F13F4E287F4F488E936F141D71351050
2016-01-14 18:31:21,714 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050]
2016-01-14 18:31:21,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796281427_0001_00_000001, stoppedTime=1452796281714, exitStatus=0
2016-01-14 18:31:21,714 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:21,714 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:21,715 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:TASK_STARTED]: vertexName=F13F4E287F4F488E936F141D71351050, taskId=task_1452796281427_0001_1_01_000000, scheduledTime=1452796281714, launchTime=1452796281714
2016-01-14 18:31:21,715 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796281427_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:21,715 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050]
2016-01-14 18:31:21,715 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:21,715 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796281427_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:21,716 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:21,716 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:21,716 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: F13F4E287F4F488E936F141D71351050, VertexParallelism: 1, TaskAttemptID:attempt_1452796281427_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A70A8C218720443BABD1638138CF317E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C20AA770125847EA8B143E49466936BC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:21,717 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796281427_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:21,717 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:21,717 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796281427_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:21,719 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796281427_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:21,719 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796281427_0001_00_000002
2016-01-14 18:31:21,719 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:21,720 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796281427_0001_00_000001, Exiting
2016-01-14 18:31:21,720 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796281427_0001_00_000001
2016-01-14 18:31:21,720 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796281427_0001_00_000001
2016-01-14 18:31:21,722 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796281427_0001_00_000002
2016-01-14 18:31:21,723 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796281427_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:21,723 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796281427_0001_00_000001 completed successfully
2016-01-14 18:31:21,723 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:21,724 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:21,726 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:21,727 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:21,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796281427_0001_00_000002, launchTime=1452796281726
2016-01-14 18:31:21,728 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796281427_0001_00_000002
2016-01-14 18:31:21,729 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796281427_0001_1_01_000000_0] to container: [container_1452796281427_0001_00_000002]
2016-01-14 18:31:21,729 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:21,730 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796281427_0001_1_01_000000_0] started. Is using containerId: [container_1452796281427_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:21,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F13F4E287F4F488E936F141D71351050, taskAttemptId=attempt_1452796281427_0001_1_01_000000_0, startTime=1452796281730, containerId=container_1452796281427_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796281427_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:21,730 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796281427_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:21,730 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796281427_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:21,731 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:21,731 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796281427_0001_00_000002 given task: attempt_1452796281427_0001_1_01_000000_0
2016-01-14 18:31:21,731 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796281427_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796281427_0001_1_01_000000_0
2016-01-14 18:31:21 Starting to run new task attempt: attempt_1452796281427_0001_1_01_000000_0
2016-01-14 18:31:21,731 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796281427_0001_1_01_000000_0
2016-01-14 18:31:21,732 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:21,732 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:21,732 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:21,732 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:21,732 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:21,732 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:21,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409] every/(1/1) ...yplatform/-1_UNKNOWN_SWAP, VertexName: F13F4E287F4F488E936F141D71351050, VertexParallelism: 1, TaskAttemptID:attempt_1452796281427_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A70A8C218720443BABD1638138CF317E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C20AA770125847EA8B143E49466936BC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:21,733 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:21,734 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796281427_0001_1_01_000000_0
2016-01-14 18:31:21,734 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:21,735 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796281427_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:21,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:21,736 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A70A8C218720443BABD1638138CF317E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:21,736 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C20AA770125847EA8B143E49466936BC, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:21,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:21,736 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:21,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:21,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C20AA770125847EA8B143E49466936BC
2016-01-14 18:31:21,738 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:21,738 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C20AA770125847EA8B143E49466936BC
2016-01-14 18:31:21,742 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:21,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A70A8C218720443BABD1638138CF317E
2016-01-14 18:31:21,745 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:21,745 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:21,746 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A70A8C218720443BABD1638138CF317E
2016-01-14 18:31:21,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A70A8C218720443BABD1638138CF317E
2016-01-14 18:31:21,751 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:21,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C20AA770125847EA8B143E49466936BC
2016-01-14 18:31:21,752 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F13F4E287F4F488E936F141D71351050, ordinal: 1
2016-01-14 18:31:21,752 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F13F4E287F4F488E936F141D71351050, mem on start (mb), free: 93, total: 489, max: 672
2016-01-14 18:31:21,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:21,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:21,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:21,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:21,753 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:21,754 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:21,754 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:21,754 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:21,755 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:21,755 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:21,756 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C20AA770125847EA8B143E49466936BC, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:21,756 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A70A8C218720443BABD1638138CF317E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:21,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:21,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A70A8C218720443BABD1638138CF317E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:21,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A70A8C218720443BABD1638138CF317E
2016-01-14 18:31:21,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:21,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:21,758 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:21,759 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:21,760 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A70A8C218720443BABD1638138CF317E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:21,760 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:21,760 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:21,760 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:21,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A70A8C218720443BABD1638138CF317E
2016-01-14 18:31:21,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:21,763 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796281427_0001_1_01_000000_0
2016-01-14 18:31:21,765 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796281427_0001_1_00_000000_0_10057, runDuration: 54239, ]
2016-01-14 18:31:21,767 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796281427_0001_1_00_000000_0_10057] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796281766, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:21,767 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A70A8C218720443BABD1638138CF317E
2016-01-14 18:31:21,767 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:21,767 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A70A8C218720443BABD1638138CF317E] #10 in 1ms
2016-01-14 18:31:21,768 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:21,769 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A70A8C218720443BABD1638138CF317E] #10, status:false, isInterrupted:false
2016-01-14 18:31:21,770 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 57DEDEE8A3CD4EB6818776917B7BCE93
2016-01-14 18:31:21,771 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP"], id: C20AA770125847EA8B143E49466936BC
2016-01-14 18:31:21,771 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_UNKNOWN_SWAP"]
2016-01-14 18:31:21,770 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:21,771 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 57DEDEE8A3CD4EB6818776917B7BCE93, for 1 inputs
2016-01-14 18:31:21,772 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:21,772 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:21,772 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:21,773 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:21,773 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A70A8C218720443BABD1638138CF317E
2016-01-14 18:31:21,773 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:21,773 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F13F4E287F4F488E936F141D71351050, all 1 inputs ready in: 00:00:00.001
2016-01-14 18:31:21,774 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1b11549c; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@ec9dba6
2016-01-14 18:31:21,778 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:21,779 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:21,779 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796281427_0001_1_01_000000_0
2016-01-14 18:31:21,780 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796281427_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:21,781 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796281427_0001_1_01_000000_0
2016-01-14 18:31:21,781 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F13F4E287F4F488E936F141D71351050, mem on close (mb), free: 90, total: 489, max: 672
2016-01-14 18:31:21,781 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A70A8C218720443BABD1638138CF317E
2016-01-14 18:31:21,782 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:21,782 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:21,782 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:21,783 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:21,783 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:21 Completed running task attempt: attempt_1452796281427_0001_1_01_000000_0
2016-01-14 18:31:21,785 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796281427_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:21,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513277952, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796281773, Process_Duration=8, Process_End_Time=1452796281781, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:21,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F13F4E287F4F488E936F141D71351050, taskAttemptId=attempt_1452796281427_0001_1_01_000000_0, startTime=1452796281730, finishTime=1452796281787, timeTaken=57, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513277952, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796281773, Process_Duration=8, Process_End_Time=1452796281781, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:21,787 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796281427_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:21,787 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796281427_0001_1_01_000000_0
2016-01-14 18:31:21,788 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:21,788 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:21,788 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796281427_0001_00_000002
2016-01-14 18:31:21,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:TASK_FINISHED]: vertexName=F13F4E287F4F488E936F141D71351050, taskId=task_1452796281427_0001_1_01_000000, startTime=1452796281730, finishTime=1452796281788, timeTaken=58, status=SUCCEEDED, successfulAttemptID=attempt_1452796281427_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513277952, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796281773, Process_Duration=8, Process_End_Time=1452796281781, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:21,789 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796281427_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:21,789 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050] : 1
2016-01-14 18:31:21,789 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050]
2016-01-14 18:31:21,790 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:VERTEX_FINISHED]: vertexName=F13F4E287F4F488E936F141D71351050, vertexId=vertex_1452796281427_0001_1_01, initRequestedTime=1452796281604, initedTime=1452796281614, startRequestedTime=1452796281631, startedTime=1452796281631, finishTime=1452796281789, timeTaken=158, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=513277952, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796281773, Process_Duration=8, Process_End_Time=1452796281781, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796281730, firstTasksToStart=[ task_1452796281427_0001_1_01_000000 ], lastTaskFinishTime=1452796281787, lastTasksToFinish=[ task_1452796281427_0001_1_01_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796281427_0001_1_01_000000 ], longestDurationTasks=[ task_1452796281427_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:21,791 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:21,791 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:21,791 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796281427_0001_1
2016-01-14 18:31:21,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796281427_0001_1
2016-01-14 18:31:21,791 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:21,791 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:21,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:21,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:21,795 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C20AA770125847EA8B143E49466936BC for vertex: vertex_1452796281427_0001_1_01 [F13F4E287F4F488E936F141D71351050]
2016-01-14 18:31:21,804 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796281427_0001_1_00 [A70A8C218720443BABD1638138CF317E]
2016-01-14 18:31:21,807 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796281427_0001_1, queueSize=0
2016-01-14 18:31:21,807 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:21,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796281427_0001_1, startTime=1452796281601, finishTime=1452796281804, timeTaken=203, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=520, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1026555904, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592563468, Process_Duration=11, Process_End_Time=2905592563479, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:21,808 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796281427_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:21,809 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796281427_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:21,809 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:21,810 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:21,810 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796281427_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796281427_0001_00_000002, stoppedTime=1452796281810, exitStatus=0
2016-01-14 18:31:21,811 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:21 Completed Dag: dag_1452796281427_0001_1
2016-01-14 18:31:21,812 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796281427_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:21,812 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796281427_0001_1_post
2016-01-14 18:31:21,813 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:21,813 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:21,814 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796281427_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:21,814 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:21,817 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_UNKNOWN_SWAP, applicationId=application_1452796281427_0001
2016-01-14 18:31:21,819 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796281427_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:21,819 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796281427_0001_00_000002: 30 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:21,820 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796281427_0001_00_000002, Exiting
2016-01-14 18:31:21,820 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:21,820 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:21,820 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:21,820 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:21,820 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:21,820 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796281427_0001_00_000002
2016-01-14 18:31:21,821 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796281427_0001_00_000002
2016-01-14 18:31:21,822 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:21,824 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796281427_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:21,824 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796281427_0001_00_000002 completed successfully
2016-01-14 18:31:21,824 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796281427_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:21,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:21,825 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:21,825 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:21,825 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:21,825 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:21,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:21,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:21,830 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/04B7D66937594517A6BA968086632F74/FAA874F56CF7401397EBF6929B00C409/.tez/application_1452796281427_0001
2016-01-14 18:31:21,831 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.433
2016-01-14 18:31:21,920 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_UNKNOWN_SWAP
2016-01-14 18:31:21,946 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:21,947 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:21,992 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 18:31:22,001 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-01-14 18:31:22,002 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:22,002 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:22,002 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:22,019 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:22,026 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1
2016-01-14 18:31:22,303 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:22,304 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:22,307 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:22,308 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:22,308 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1"]
2016-01-14 18:31:22,308 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:22,308 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:22,308 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:22,309 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 18:31:22,310 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:22,323 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:22,328 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:22,328 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:22,328 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:22,337 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E/.tez/application_1452796282328_0001 doesn't exist and is created
2016-01-14 18:31:22,351 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:22,351 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:22,351 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E/.tez/application_1452796282328_0001
2016-01-14 18:31:22,354 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796282328_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:22,365 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:22,366 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:22,368 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:22,368 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:22,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E/.tez/application_1452796282328_0001/localmode-log-dir
2016-01-14 18:31:22,369 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E/.tez/application_1452796282328_0001/localmode-log-dir/history.txt.appattempt_1452796282328_0001_000000, maxErrors=10
2016-01-14 18:31:22,369 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:22,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796282328_0001_000000, appSubmitTime=1452796282353, launchTime=1452796282353
2016-01-14 18:31:22,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:22,376 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:22,376 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:22,379 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:22,379 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:33259
2016-01-14 18:31:22,383 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:22,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796282328_0001_000000, startTime=1452796282385
2016-01-14 18:31:22,385 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:22,385 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:22,451 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:22,451 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:22,451 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1452796282328_0001, dagName=[6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 18:31:22,476 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 18:31:22,477 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796282328_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E/.tez/application_1452796282328_0001/localmode-log-dir/dag_1452796282328_0001_1.dot
2016-01-14 18:31:22,478 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E/.tez/application_1452796282328_0001/localmode-log-dir/dag_1452796282328_0001_1-tez-dag.pb.txt
2016-01-14 18:31:22,486 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796282328_0001_1
2016-01-14 18:31:22,486 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:22,486 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 18:31:22 Running Dag: dag_1452796282328_0001_1
2016-01-14 18:31:22,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796282328_0001_1, submitTime=1452796282476
2016-01-14 18:31:22,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:22,496 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E5A0AD41998F45E9933F1659975BB96B
2016-01-14 18:31:22,497 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:22,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:22,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796282328_0001_1, initTime=1452796282496
2016-01-14 18:31:22,498 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796282328_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:22,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:DAG_STARTED]: dagID=dag_1452796282328_0001_1, startTime=1452796282499
2016-01-14 18:31:22,500 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:22,500 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796282328_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:22,500 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1452796282328_0001, dagName=[6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E] every/(1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 18:31:22,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:22,500 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D6C655E8F0A84F8692509012FC8F6FBC : {659CC6C74A8941818033377D00CE8FC3={InputName=659CC6C74A8941818033377D00CE8FC3}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:22,500 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 659CC6C74A8941818033377D00CE8FC3, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:22,501 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC]
2016-01-14 18:31:22,501 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC]
2016-01-14 18:31:22,501 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC]
2016-01-14 18:31:22,501 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC]. Starting root input initializers: 1
2016-01-14 18:31:22,502 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:22,502 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B]
2016-01-14 18:31:22,502 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 659CC6C74A8941818033377D00CE8FC3 on vertex vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC]
2016-01-14 18:31:22,504 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:22,504 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 659CC6C74A8941818033377D00CE8FC3 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:22,505 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:22,505 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B]
2016-01-14 18:31:22,505 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:22,505 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B]
2016-01-14 18:31:22,507 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B]
2016-01-14 18:31:22,507 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=BC9C537C9A1D4A269BA31A65FAAA1A32, vertexId=vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:22,507 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=BC9C537C9A1D4A269BA31A65FAAA1A32, vertexId=vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B]
2016-01-14 18:31:22,506 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:22,511 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:22,511 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 659CC6C74A8941818033377D00CE8FC3 on vertex vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC]
2016-01-14 18:31:22,512 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=BC9C537C9A1D4A269BA31A65FAAA1A32, vertexName=E5A0AD41998F45E9933F1659975BB96B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:22,513 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=BC9C537C9A1D4A269BA31A65FAAA1A32, vertexId=vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B]
2016-01-14 18:31:22,514 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:22,514 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E5A0AD41998F45E9933F1659975BB96B, vertexId=vertex_1452796282328_0001_1_01, initRequestedTime=1452796282502, initedTime=1452796282513, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:22,514 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:22,514 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:22,514 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {659CC6C74A8941818033377D00CE8FC3=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:22,515 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC] parallelism set to 1
2016-01-14 18:31:22,515 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:22,515 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796282328_0001_1_00, eventCount=1
2016-01-14 18:31:22,516 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:22,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D6C655E8F0A84F8692509012FC8F6FBC, vertexId=vertex_1452796282328_0001_1_00, initRequestedTime=1452796282500, initedTime=1452796282515, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:22,516 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC] with distanceFromRoot: 0
2016-01-14 18:31:22,516 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:22,517 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D6C655E8F0A84F8692509012FC8F6FBC
2016-01-14 18:31:22,517 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC]
2016-01-14 18:31:22,517 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:22,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796282328_0001_1_00, startRequestedTime=1452796282517, startedTime=1452796282517
2016-01-14 18:31:22,518 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:22,519 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:22,519 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:TASK_STARTED]: vertexName=D6C655E8F0A84F8692509012FC8F6FBC, taskId=task_1452796282328_0001_1_00_000000, scheduledTime=1452796282518, launchTime=1452796282518
2016-01-14 18:31:22,519 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282328_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:22,519 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796282328_0001_1_00 for vertex: vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] numStartedSources: 1 numSources: 1
2016-01-14 18:31:22,520 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] with distanceFromRoot: 1
2016-01-14 18:31:22,520 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC]
2016-01-14 18:31:22,520 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D6C655E8F0A84F8692509012FC8F6FBC in vertex: E5A0AD41998F45E9933F1659975BB96B
2016-01-14 18:31:22,521 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E5A0AD41998F45E9933F1659975BB96B with 1 source tasks and 1 pending tasks
2016-01-14 18:31:22,521 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796282328_0001, with dag id: dag_1452796282328_0001_1
2016-01-14 18:31:22,521 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:22,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796282328_0001_1_01, startRequestedTime=1452796282520, startedTime=1452796282520
2016-01-14 18:31:22,525 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:22,525 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: D6C655E8F0A84F8692509012FC8F6FBC, VertexParallelism: 1, TaskAttemptID:attempt_1452796282328_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=659CC6C74A8941818033377D00CE8FC3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E5A0AD41998F45E9933F1659975BB96B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:22,525 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282328_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:22,526 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:22,526 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:22,527 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:22,527 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:22,527 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796282328_0001_00_000001
2016-01-14 18:31:22,527 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:22,532 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796282328_0001_00_000001
2016-01-14 18:31:22,532 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:22,533 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:22,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:22,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796282328_0001_00_000001, launchTime=1452796282532
2016-01-14 18:31:22,534 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796282328_0001_00_000001
2016-01-14 18:31:22,534 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796282328_0001_1_00_000000_0] to container: [container_1452796282328_0001_00_000001]
2016-01-14 18:31:22,534 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:22,535 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796282328_0001_1_00_000000_0] started. Is using containerId: [container_1452796282328_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:22,535 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D6C655E8F0A84F8692509012FC8F6FBC, taskAttemptId=attempt_1452796282328_0001_1_00_000000_0, startTime=1452796282535, containerId=container_1452796282328_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796282328_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:22,535 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282328_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:22,535 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282328_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:22,536 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:22,536 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796282328_0001_00_000001 given task: attempt_1452796282328_0001_1_00_000000_0
2016-01-14 18:31:22,536 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796282328_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796282328_0001_1_00_000000_0
2016-01-14 18:31:22 Starting to run new task attempt: attempt_1452796282328_0001_1_00_000000_0
2016-01-14 18:31:22,537 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796282328_0001_1_00_000000_0
2016-01-14 18:31:22,537 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:22,537 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:22,537 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:22,537 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:22,537 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:22,537 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:22,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: D6C655E8F0A84F8692509012FC8F6FBC, VertexParallelism: 1, TaskAttemptID:attempt_1452796282328_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=659CC6C74A8941818033377D00CE8FC3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E5A0AD41998F45E9933F1659975BB96B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:22,538 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:22,538 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796282328_0001_1_00_000000_0
2016-01-14 18:31:22,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:22,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:22,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=659CC6C74A8941818033377D00CE8FC3, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:22,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E5A0AD41998F45E9933F1659975BB96B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:22,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:22,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:22,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 659CC6C74A8941818033377D00CE8FC3
2016-01-14 18:31:22,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:22,541 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E5A0AD41998F45E9933F1659975BB96B
2016-01-14 18:31:22,541 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796282328_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:22,541 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 659CC6C74A8941818033377D00CE8FC3
2016-01-14 18:31:22,543 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:22,543 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:22,544 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E5A0AD41998F45E9933F1659975BB96B
2016-01-14 18:31:22,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E5A0AD41998F45E9933F1659975BB96B
2016-01-14 18:31:22,546 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:22,546 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 659CC6C74A8941818033377D00CE8FC3
2016-01-14 18:31:22,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 659CC6C74A8941818033377D00CE8FC3
2016-01-14 18:31:22,549 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D6C655E8F0A84F8692509012FC8F6FBC, ordinal: 0
2016-01-14 18:31:22,549 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D6C655E8F0A84F8692509012FC8F6FBC, mem on start (mb), free: 286, total: 368, max: 672
2016-01-14 18:31:22,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:22,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:22,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:22,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:22,551 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:22,551 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:22,551 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:22,551 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:22,552 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:22,552 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:22,552 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 659CC6C74A8941818033377D00CE8FC3, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:22,552 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E5A0AD41998F45E9933F1659975BB96B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:22,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:22,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 659CC6C74A8941818033377D00CE8FC3 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:22,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:22,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:22,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 659CC6C74A8941818033377D00CE8FC3
2016-01-14 18:31:22,553 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:22,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 659CC6C74A8941818033377D00CE8FC3
2016-01-14 18:31:22,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:22,554 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796282328_0001_1_00_000000_0
2016-01-14 18:31:22,555 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:22,555 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@32d94550
2016-01-14 18:31:22,556 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@32d94550
2016-01-14 18:31:22,556 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:22,557 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:22,557 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:22,558 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 659CC6C74A8941818033377D00CE8FC3
2016-01-14 18:31:22,558 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 4E36809706B04DD3A2870447AC470084
2016-01-14 18:31:22,559 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 4E36809706B04DD3A2870447AC470084
2016-01-14 18:31:22,559 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@a88cdff; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1b8c25c0; comparator=cascading.tuple.hadoop.util.TupleComparator@e026e61; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:22,560 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:22,560 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:22,560 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:22,573 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:22,573 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:22,573 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:22,573 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:22,574 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:22,575 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:22,575 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:22,576 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D6C655E8F0A84F8692509012FC8F6FBC, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:22,578 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796282328_0001_1_00_000000_0
2016-01-14 18:31:22,579 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D6C655E8F0A84F8692509012FC8F6FBC, mem on close (mb), free: 184, total: 368, max: 672
2016-01-14 18:31:22,579 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:22,579 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:22,579 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:22,579 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:22,582 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:22 Completed running task attempt: attempt_1452796282328_0001_1_00_000000_0
2016-01-14 18:31:22,585 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796282328_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:22,586 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796282328_0001_1_00, eventCount=1
2016-01-14 18:31:22,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:22,586 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D6C655E8F0A84F8692509012FC8F6FBC, taskAttemptId=attempt_1452796282328_0001_1_00_000000_0, startTime=1452796282535, finishTime=1452796282586, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796282576, Process_Duration=2, Process_End_Time=1452796282578, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:22,586 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282328_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:22,587 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796282328_0001_1_00_000000_0
2016-01-14 18:31:22,587 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:22,587 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:22,587 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:TASK_FINISHED]: vertexName=D6C655E8F0A84F8692509012FC8F6FBC, taskId=task_1452796282328_0001_1_00_000000, startTime=1452796282535, finishTime=1452796282587, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1452796282328_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796282576, Process_Duration=2, Process_End_Time=1452796282578, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:22,588 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282328_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:22,588 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC] : 1
2016-01-14 18:31:22,588 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796282576, Process_Duration=2, Process_End_Time=1452796282578, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:22,589 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:22,589 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:22,589 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796282328_0001_00_000001
2016-01-14 18:31:22,588 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC]
2016-01-14 18:31:22,590 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:VERTEX_FINISHED]: vertexName=D6C655E8F0A84F8692509012FC8F6FBC, vertexId=vertex_1452796282328_0001_1_00, initRequestedTime=1452796282500, initedTime=1452796282515, startRequestedTime=1452796282517, startedTime=1452796282517, finishTime=1452796282589, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796282576, Process_Duration=2, Process_End_Time=1452796282578, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796282535, firstTasksToStart=[ task_1452796282328_0001_1_00_000000 ], lastTaskFinishTime=1452796282586, lastTasksToFinish=[ task_1452796282328_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796282328_0001_1_00_000000 ], longestDurationTasks=[ task_1452796282328_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:22,590 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:22,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:22,591 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:22,591 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796282328_0001_00_000001, stoppedTime=1452796282591, exitStatus=0
2016-01-14 18:31:22,591 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:22,591 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] attempt: attempt_1452796282328_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:22,591 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E5A0AD41998F45E9933F1659975BB96B
2016-01-14 18:31:22,591 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B]
2016-01-14 18:31:22,592 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:22,593 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796282328_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:22,593 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:TASK_STARTED]: vertexName=E5A0AD41998F45E9933F1659975BB96B, taskId=task_1452796282328_0001_1_01_000000, scheduledTime=1452796282593, launchTime=1452796282593
2016-01-14 18:31:22,593 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282328_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:22,593 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B]
2016-01-14 18:31:22,593 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:22,593 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: E5A0AD41998F45E9933F1659975BB96B, VertexParallelism: 1, TaskAttemptID:attempt_1452796282328_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D6C655E8F0A84F8692509012FC8F6FBC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BC9C537C9A1D4A269BA31A65FAAA1A32, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:22,593 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282328_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:22,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:22,594 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:22,594 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:22,594 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796282328_0001_00_000002
2016-01-14 18:31:22,595 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:22,597 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796282328_0001_00_000002
2016-01-14 18:31:22,597 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:22,597 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:22,598 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:22,598 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796282328_0001_00_000002, launchTime=1452796282598
2016-01-14 18:31:22,599 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796282328_0001_00_000002
2016-01-14 18:31:22,599 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796282328_0001_1_01_000000_0] to container: [container_1452796282328_0001_00_000002]
2016-01-14 18:31:22,599 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:22,599 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796282328_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:22,599 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796282328_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:22,600 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796282328_0001_00_000002 given task: attempt_1452796282328_0001_1_01_000000_0
2016-01-14 18:31:22,600 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796282328_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796282328_0001_1_01_000000_0
2016-01-14 18:31:22,600 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796282328_0001_00_000001, Exiting
2016-01-14 18:31:22,600 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796282328_0001_00_000001
2016-01-14 18:31:22,600 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796282328_0001_1_01_000000_0] started. Is using containerId: [container_1452796282328_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:22,601 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E5A0AD41998F45E9933F1659975BB96B, taskAttemptId=attempt_1452796282328_0001_1_01_000000_0, startTime=1452796282600, containerId=container_1452796282328_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796282328_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:22,601 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796282328_0001_00_000001
2016-01-14 18:31:22,601 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:22,601 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282328_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:22 Starting to run new task attempt: attempt_1452796282328_0001_1_01_000000_0
2016-01-14 18:31:22,602 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282328_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:22,602 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796282328_0001_1_01_000000_0
2016-01-14 18:31:22,602 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:22,602 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:22,603 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:22,603 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:22,604 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:22,604 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:22,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E] every/(1/1) ...mblyplatform/-1_VALUES_-1, VertexName: E5A0AD41998F45E9933F1659975BB96B, VertexParallelism: 1, TaskAttemptID:attempt_1452796282328_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D6C655E8F0A84F8692509012FC8F6FBC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BC9C537C9A1D4A269BA31A65FAAA1A32, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:22,604 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:22,603 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796282328_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:22,605 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796282328_0001_00_000001 completed successfully
2016-01-14 18:31:22,605 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:22,605 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796282328_0001_1_01_000000_0
2016-01-14 18:31:22,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:22,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:22,607 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796282328_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:22,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:22,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D6C655E8F0A84F8692509012FC8F6FBC, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:22,608 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:22,608 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D6C655E8F0A84F8692509012FC8F6FBC
2016-01-14 18:31:22,608 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BC9C537C9A1D4A269BA31A65FAAA1A32, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:22,609 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:22,609 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BC9C537C9A1D4A269BA31A65FAAA1A32
2016-01-14 18:31:22,609 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:22,610 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: BC9C537C9A1D4A269BA31A65FAAA1A32
2016-01-14 18:31:22,610 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:22,610 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:22,611 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:22,611 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D6C655E8F0A84F8692509012FC8F6FBC
2016-01-14 18:31:22,612 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D6C655E8F0A84F8692509012FC8F6FBC
2016-01-14 18:31:22,618 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:22,619 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BC9C537C9A1D4A269BA31A65FAAA1A32
2016-01-14 18:31:22,620 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E5A0AD41998F45E9933F1659975BB96B, ordinal: 1
2016-01-14 18:31:22,620 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5A0AD41998F45E9933F1659975BB96B, mem on start (mb), free: 177, total: 368, max: 672
2016-01-14 18:31:22,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:22,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:22,622 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:22,622 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:22,623 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:22,623 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:22,624 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:22,624 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:22,624 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:22,625 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:22,625 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BC9C537C9A1D4A269BA31A65FAAA1A32, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:22,626 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D6C655E8F0A84F8692509012FC8F6FBC, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:22,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:22,627 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D6C655E8F0A84F8692509012FC8F6FBC
2016-01-14 18:31:22,627 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:22,628 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:22,628 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D6C655E8F0A84F8692509012FC8F6FBC with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:22,628 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:22,628 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:22,628 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:22,629 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D6C655E8F0A84F8692509012FC8F6FBC being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:22,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D6C655E8F0A84F8692509012FC8F6FBC
2016-01-14 18:31:22,630 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:22,631 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:22,631 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:22,631 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796282328_0001_1_01_000000_0
2016-01-14 18:31:22,631 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796282328_0001_1_00_000000_0_10063, runDuration: 41545, ]
2016-01-14 18:31:22,633 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796282328_0001_1_00_000000_0_10063] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796282633, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:22,634 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D6C655E8F0A84F8692509012FC8F6FBC
2016-01-14 18:31:22,635 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:22,636 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:22,637 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D6C655E8F0A84F8692509012FC8F6FBC] #11 in 4ms
2016-01-14 18:31:22,637 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 4E36809706B04DD3A2870447AC470084
2016-01-14 18:31:22,638 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1"], id: BC9C537C9A1D4A269BA31A65FAAA1A32
2016-01-14 18:31:22,638 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_-1"]
2016-01-14 18:31:22,638 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 4E36809706B04DD3A2870447AC470084, for 1 inputs
2016-01-14 18:31:22,640 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D6C655E8F0A84F8692509012FC8F6FBC] #11, status:false, isInterrupted:false
2016-01-14 18:31:22,641 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:22,641 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:22,641 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:22,642 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:22,642 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:22,642 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D6C655E8F0A84F8692509012FC8F6FBC
2016-01-14 18:31:22,642 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:22,642 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E5A0AD41998F45E9933F1659975BB96B, all 1 inputs ready in: 00:00:00.003
2016-01-14 18:31:22,645 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@45bbbb4a; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@28051b0a
2016-01-14 18:31:22,646 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:22,646 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:22,648 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:22,648 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:22,649 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796282328_0001_1_01_000000_0
2016-01-14 18:31:22,650 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796282328_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:22,651 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796282328_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:22,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796282643, Process_Duration=8, Process_End_Time=1452796282651, Read_Duration=1, Tuples_Read=1]]
2016-01-14 18:31:22,654 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:22,654 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E5A0AD41998F45E9933F1659975BB96B, taskAttemptId=attempt_1452796282328_0001_1_01_000000_0, startTime=1452796282600, finishTime=1452796282654, timeTaken=54, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796282643, Process_Duration=8, Process_End_Time=1452796282651, Read_Duration=1, Tuples_Read=1
2016-01-14 18:31:22,654 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282328_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:22,655 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796282328_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:22,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:22,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:TASK_FINISHED]: vertexName=E5A0AD41998F45E9933F1659975BB96B, taskId=task_1452796282328_0001_1_01_000000, startTime=1452796282600, finishTime=1452796282655, timeTaken=55, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:22,655 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282328_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:22,655 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] : 1
2016-01-14 18:31:22,655 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] because task failed: task_1452796282328_0001_1_01_000000
2016-01-14 18:31:22,656 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:22,655 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:22,656 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:22,657 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B]
2016-01-14 18:31:22,658 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=BC9C537C9A1D4A269BA31A65FAAA1A32, vertexId=vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B]
2016-01-14 18:31:22,659 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:22,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:VERTEX_FINISHED]: vertexName=E5A0AD41998F45E9933F1659975BB96B, vertexId=vertex_1452796282328_0001_1_01, initRequestedTime=1452796282502, initedTime=1452796282513, startRequestedTime=1452796282520, startedTime=1452796282520, finishTime=1452796282659, timeTaken=139, status=FAILED, diagnostics=Task failed, taskId=task_1452796282328_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796282600, firstTasksToStart=[ task_1452796282328_0001_1_01_000000 ], lastTaskFinishTime=1452796282654, lastTasksToFinish=[ task_1452796282328_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:22,659 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:22,660 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796282328_0001_00_000002, stoppedTime=1452796282660, exitStatus=0
2016-01-14 18:31:22,660 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:22,660 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:22,660 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796282328_0001_1_01_000000_0 failed: true container: container_1452796282328_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:22,664 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E5A0AD41998F45E9933F1659975BB96B, mem on close (mb), free: 172, total: 368, max: 672
2016-01-14 18:31:22,664 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:22,664 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:22,664 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E5A0AD41998F45E9933F1659975BB96B, index=1
2016-01-14 18:31:22,664 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796282328_0001_1
2016-01-14 18:31:22,664 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796282328_0001_1_00 [D6C655E8F0A84F8692509012FC8F6FBC]
2016-01-14 18:31:22,664 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: BC9C537C9A1D4A269BA31A65FAAA1A32 for vertex: vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B]
2016-01-14 18:31:22,666 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796282328_0001_1, queueSize=0
2016-01-14 18:31:22,667 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:22,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282328_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796282328_0001_1, startTime=1452796282499, finishTime=1452796282665, timeTaken=166, status=FAILED, diagnostics=Vertex failed, vertexName=E5A0AD41998F45E9933F1659975BB96B, vertexId=vertex_1452796282328_0001_1_01, diagnostics=[Task failed, taskId=task_1452796282328_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796282328_0001_1_01 [E5A0AD41998F45E9933F1659975BB96B] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796282576, Process_Duration=2, Process_End_Time=1452796282578, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:22,667 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796282328_0001_1 finished with state: FAILED
2016-01-14 18:31:22,667 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796282328_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:22,668 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796282328_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:22 Completed Dag: dag_1452796282328_0001_1
2016-01-14 18:31:22,668 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796282328_0001_1, dagState=FAILED
2016-01-14 18:31:22,668 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796282328_0001_1_post
2016-01-14 18:31:22,668 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:22,668 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:22,668 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:22,670 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D6C655E8F0A84F8692509012FC8F6FBC
2016-01-14 18:31:22,670 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:22,670 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:22,671 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:22,671 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:22,671 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:22,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E5A0AD41998F45E9933F1659975BB96B, sourceVertex=D6C655E8F0A84F8692509012FC8F6FBC
2016-01-14 18:31:22,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E5A0AD41998F45E9933F1659975BB96B, sourceVertex=BC9C537C9A1D4A269BA31A65FAAA1A32
2016-01-14 18:31:22,671 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796282328_0001_00_000002
2016-01-14 18:31:22,671 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796282328_0001_00_000002
2016-01-14 18:31:22,672 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796282328_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:22,672 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796282328_0001_00_000002 completed but with errors
2016-01-14 18:31:22,672 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796282328_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:22,672 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282328_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:22,672 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282328_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:22,677 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/-1_VALUES_-1, applicationId=application_1452796282328_0001
2016-01-14 18:31:22,680 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:22,680 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:22,680 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:22,680 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:22,680 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:22,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:22,683 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:22,684 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:22,683 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:22,684 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:22,684 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:22,684 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:22,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:22,689 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6975CEEACF97458D993558D078B61EE5/D456DEFAA81A48B1B97001E1874F6C6E/.tez/application_1452796282328_0001
2016-01-14 18:31:22,689 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:22,690 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...mblyplatform/-1_VALUES_-1
2016-01-14 18:31:22,690 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:22,691 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.383
2016-01-14 18:31:22,691 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_-1
2016-01-14 18:31:22,728 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:22,728 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:22,762 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.006
2016-01-14 18:31:22,765 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.009
2016-01-14 18:31:22,766 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:22,766 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:22,766 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_'field'
2016-01-14 18:31:22,790 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:22,790 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:22,833 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 18:31:22,843 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.025
2016-01-14 18:31:22,844 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:22,844 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:22,844 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:22,864 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:22,870 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS
2016-01-14 18:31:22,922 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:22,923 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:22,923 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS"]
2016-01-14 18:31:22,924 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:22,924 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:22,925 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:22,925 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 18:31:22,926 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:22,938 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:22,942 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:22,943 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:22,943 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:22,952 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E/.tez/application_1452796282942_0001 doesn't exist and is created
2016-01-14 18:31:22,967 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:22,968 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:22,969 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E/.tez/application_1452796282942_0001
2016-01-14 18:31:22,971 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796282942_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:22,982 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:22,983 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:22,984 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:22,985 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:22,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E/.tez/application_1452796282942_0001/localmode-log-dir
2016-01-14 18:31:22,985 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E/.tez/application_1452796282942_0001/localmode-log-dir/history.txt.appattempt_1452796282942_0001_000000, maxErrors=10
2016-01-14 18:31:22,985 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:22,985 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796282942_0001_000000, appSubmitTime=1452796282971, launchTime=1452796282971
2016-01-14 18:31:22,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:22,989 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:22,990 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:22,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:22,993 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:55428
2016-01-14 18:31:22,995 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:22,998 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796282942_0001_000000, startTime=1452796282998
2016-01-14 18:31:22,998 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:22,998 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:23,068 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:23,068 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:23,068 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1452796282942_0001, dagName=[F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 18:31:23,088 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 18:31:23,089 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796282942_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E/.tez/application_1452796282942_0001/localmode-log-dir/dag_1452796282942_0001_1.dot
2016-01-14 18:31:23,090 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E/.tez/application_1452796282942_0001/localmode-log-dir/dag_1452796282942_0001_1-tez-dag.pb.txt
2016-01-14 18:31:23,098 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796282942_0001_1
2016-01-14 18:31:23,098 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:23,098 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 18:31:23 Running Dag: dag_1452796282942_0001_1
2016-01-14 18:31:23,108 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796282942_0001_1, submitTime=1452796283088
2016-01-14 18:31:23,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:23,109 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F368293C2AAF42E18F0D597A146C4DE9
2016-01-14 18:31:23,109 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:23,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:23,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796282942_0001_1, initTime=1452796283108
2016-01-14 18:31:23,111 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796282942_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:23,111 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:23,111 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:DAG_STARTED]: dagID=dag_1452796282942_0001_1, startTime=1452796283111
2016-01-14 18:31:23,112 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:23,112 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796282942_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:23,112 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1452796282942_0001, dagName=[F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E] every/(1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 18:31:23,112 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B9CFE9BD304D4140AF7211026E582774 : {0E7AC0246D694BAF8F490662B62A098F={InputName=0E7AC0246D694BAF8F490662B62A098F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:23,112 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 0E7AC0246D694BAF8F490662B62A098F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:23,112 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774]
2016-01-14 18:31:23,112 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774]
2016-01-14 18:31:23,113 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774]
2016-01-14 18:31:23,113 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:23,113 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:23,113 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774]. Starting root input initializers: 1
2016-01-14 18:31:23,113 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:23,113 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9]
2016-01-14 18:31:23,114 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 0E7AC0246D694BAF8F490662B62A098F on vertex vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774]
2016-01-14 18:31:23,117 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:23,117 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:23,117 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9]
2016-01-14 18:31:23,117 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9]
2016-01-14 18:31:23,118 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9]
2016-01-14 18:31:23,118 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=66B413E1CE85475EB0382F98FF4F2149, vertexId=vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:23,118 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=66B413E1CE85475EB0382F98FF4F2149, vertexId=vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9]
2016-01-14 18:31:23,120 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=66B413E1CE85475EB0382F98FF4F2149, vertexName=F368293C2AAF42E18F0D597A146C4DE9, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:23,120 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=66B413E1CE85475EB0382F98FF4F2149, vertexId=vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9]
2016-01-14 18:31:23,117 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 0E7AC0246D694BAF8F490662B62A098F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:23,121 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:23,121 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:23,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F368293C2AAF42E18F0D597A146C4DE9, vertexId=vertex_1452796282942_0001_1_01, initRequestedTime=1452796283113, initedTime=1452796283121, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:23,121 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:23,122 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:23,123 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:23,124 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 0E7AC0246D694BAF8F490662B62A098F on vertex vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774]
2016-01-14 18:31:23,124 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:23,124 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {0E7AC0246D694BAF8F490662B62A098F=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:23,125 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774] parallelism set to 1
2016-01-14 18:31:23,125 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:23,125 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796282942_0001_1_00, eventCount=1
2016-01-14 18:31:23,126 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B9CFE9BD304D4140AF7211026E582774, vertexId=vertex_1452796282942_0001_1_00, initRequestedTime=1452796283112, initedTime=1452796283125, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:23,126 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774] with distanceFromRoot: 0
2016-01-14 18:31:23,126 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:23,127 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:23,127 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B9CFE9BD304D4140AF7211026E582774
2016-01-14 18:31:23,127 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774]
2016-01-14 18:31:23,128 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:23,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796282942_0001_1_00, startRequestedTime=1452796283127, startedTime=1452796283127
2016-01-14 18:31:23,128 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:23,129 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:23,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:TASK_STARTED]: vertexName=B9CFE9BD304D4140AF7211026E582774, taskId=task_1452796282942_0001_1_00_000000, scheduledTime=1452796283128, launchTime=1452796283128
2016-01-14 18:31:23,129 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282942_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:23,129 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796282942_0001_1_00 for vertex: vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] numStartedSources: 1 numSources: 1
2016-01-14 18:31:23,129 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] with distanceFromRoot: 1
2016-01-14 18:31:23,130 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774]
2016-01-14 18:31:23,131 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B9CFE9BD304D4140AF7211026E582774 in vertex: F368293C2AAF42E18F0D597A146C4DE9
2016-01-14 18:31:23,131 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F368293C2AAF42E18F0D597A146C4DE9 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:23,134 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796282942_0001, with dag id: dag_1452796282942_0001_1
2016-01-14 18:31:23,134 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796282942_0001_1_01, startRequestedTime=1452796283131, startedTime=1452796283131
2016-01-14 18:31:23,134 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:23,138 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:23,139 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: B9CFE9BD304D4140AF7211026E582774, VertexParallelism: 1, TaskAttemptID:attempt_1452796282942_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0E7AC0246D694BAF8F490662B62A098F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F368293C2AAF42E18F0D597A146C4DE9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:23,139 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282942_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:23,139 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:23,140 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:23,140 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:23,140 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:23,141 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796282942_0001_00_000001
2016-01-14 18:31:23,143 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:23,144 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796282942_0001_00_000001
2016-01-14 18:31:23,144 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:23,145 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:23,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:23,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796282942_0001_00_000001, launchTime=1452796283145
2016-01-14 18:31:23,146 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796282942_0001_00_000001
2016-01-14 18:31:23,147 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796282942_0001_1_00_000000_0] to container: [container_1452796282942_0001_00_000001]
2016-01-14 18:31:23,147 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:23,147 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796282942_0001_00_000001 given task: attempt_1452796282942_0001_1_00_000000_0
2016-01-14 18:31:23,148 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796282942_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796282942_0001_1_00_000000_0
2016-01-14 18:31:23,148 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796282942_0001_1_00_000000_0] started. Is using containerId: [container_1452796282942_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:23,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:23,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B9CFE9BD304D4140AF7211026E582774, taskAttemptId=attempt_1452796282942_0001_1_00_000000_0, startTime=1452796283147, containerId=container_1452796282942_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796282942_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:23,148 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282942_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:23,149 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282942_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:23 Starting to run new task attempt: attempt_1452796282942_0001_1_00_000000_0
2016-01-14 18:31:23,149 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796282942_0001_1_00_000000_0
2016-01-14 18:31:23,149 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:23,149 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:23,150 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:23,150 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:23,150 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:23,150 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:23,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: B9CFE9BD304D4140AF7211026E582774, VertexParallelism: 1, TaskAttemptID:attempt_1452796282942_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=0E7AC0246D694BAF8F490662B62A098F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F368293C2AAF42E18F0D597A146C4DE9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:23,151 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:23,151 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796282942_0001_1_00_000000_0
2016-01-14 18:31:23,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:23,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,152 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796282942_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:23,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=0E7AC0246D694BAF8F490662B62A098F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:23,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F368293C2AAF42E18F0D597A146C4DE9, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:23,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,153 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:23,153 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:23,153 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 0E7AC0246D694BAF8F490662B62A098F
2016-01-14 18:31:23,154 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 0E7AC0246D694BAF8F490662B62A098F
2016-01-14 18:31:23,153 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F368293C2AAF42E18F0D597A146C4DE9
2016-01-14 18:31:23,156 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:23,158 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F368293C2AAF42E18F0D597A146C4DE9
2016-01-14 18:31:23,158 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:23,158 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:23,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F368293C2AAF42E18F0D597A146C4DE9
2016-01-14 18:31:23,158 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 0E7AC0246D694BAF8F490662B62A098F
2016-01-14 18:31:23,159 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 0E7AC0246D694BAF8F490662B62A098F
2016-01-14 18:31:23,164 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B9CFE9BD304D4140AF7211026E582774, ordinal: 0
2016-01-14 18:31:23,164 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9CFE9BD304D4140AF7211026E582774, mem on start (mb), free: 272, total: 369, max: 672
2016-01-14 18:31:23,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:23,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:23,164 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:23,164 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:23,164 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:23,164 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:23,164 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:23,165 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:23,165 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:23,165 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 0E7AC0246D694BAF8F490662B62A098F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:23,165 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F368293C2AAF42E18F0D597A146C4DE9, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:23,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:23,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 0E7AC0246D694BAF8F490662B62A098F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:23,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 0E7AC0246D694BAF8F490662B62A098F
2016-01-14 18:31:23,165 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:23,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 0E7AC0246D694BAF8F490662B62A098F
2016-01-14 18:31:23,165 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:23,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:23,166 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:23,166 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796282942_0001_1_00_000000_0
2016-01-14 18:31:23,166 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:23,167 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3f5070f2
2016-01-14 18:31:23,168 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3f5070f2
2016-01-14 18:31:23,168 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:23,168 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:23,170 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:23,170 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 0E7AC0246D694BAF8F490662B62A098F
2016-01-14 18:31:23,170 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 8021D5BC24344B309EFA50E0028DCAAC
2016-01-14 18:31:23,171 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 8021D5BC24344B309EFA50E0028DCAAC
2016-01-14 18:31:23,171 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@3825350; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@612baafc; comparator=cascading.tuple.hadoop.util.TupleComparator@79e6b83a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:23,171 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:23,172 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:23,172 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:23,184 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:23,185 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:23,185 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:23,185 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:23,185 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:23,187 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:23,187 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:23,188 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B9CFE9BD304D4140AF7211026E582774, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:23,192 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796282942_0001_1_00_000000_0
2016-01-14 18:31:23,193 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B9CFE9BD304D4140AF7211026E582774, mem on close (mb), free: 170, total: 369, max: 672
2016-01-14 18:31:23,193 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:23,193 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:23,193 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:23,193 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:23,196 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:23 Completed running task attempt: attempt_1452796282942_0001_1_00_000000_0
2016-01-14 18:31:23,198 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796282942_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:23,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796282942_0001_1_00, eventCount=1
2016-01-14 18:31:23,199 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796283188, Process_Duration=4, Process_End_Time=1452796283192, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:23,199 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:23,199 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:23,199 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:23,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B9CFE9BD304D4140AF7211026E582774, taskAttemptId=attempt_1452796282942_0001_1_00_000000_0, startTime=1452796283147, finishTime=1452796283199, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796283188, Process_Duration=4, Process_End_Time=1452796283192, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:23,200 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282942_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:23,200 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796282942_0001_1_00_000000_0
2016-01-14 18:31:23,200 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796282942_0001_00_000001
2016-01-14 18:31:23,200 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:23,201 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:TASK_FINISHED]: vertexName=B9CFE9BD304D4140AF7211026E582774, taskId=task_1452796282942_0001_1_00_000000, startTime=1452796283147, finishTime=1452796283200, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1452796282942_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796283188, Process_Duration=4, Process_End_Time=1452796283192, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:23,201 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282942_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:23,201 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774] : 1
2016-01-14 18:31:23,201 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774]
2016-01-14 18:31:23,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:23,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:23,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:VERTEX_FINISHED]: vertexName=B9CFE9BD304D4140AF7211026E582774, vertexId=vertex_1452796282942_0001_1_00, initRequestedTime=1452796283112, initedTime=1452796283125, startRequestedTime=1452796283127, startedTime=1452796283127, finishTime=1452796283202, timeTaken=75, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796283188, Process_Duration=4, Process_End_Time=1452796283192, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796283147, firstTasksToStart=[ task_1452796282942_0001_1_00_000000 ], lastTaskFinishTime=1452796283199, lastTasksToFinish=[ task_1452796282942_0001_1_00_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796282942_0001_1_00_000000 ], longestDurationTasks=[ task_1452796282942_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:23,203 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:23,203 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:23,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796282942_0001_00_000001, stoppedTime=1452796283203, exitStatus=0
2016-01-14 18:31:23,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:23,204 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:23,209 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] attempt: attempt_1452796282942_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:23,210 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F368293C2AAF42E18F0D597A146C4DE9
2016-01-14 18:31:23,210 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9]
2016-01-14 18:31:23,210 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:23,210 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796282942_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:23,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:TASK_STARTED]: vertexName=F368293C2AAF42E18F0D597A146C4DE9, taskId=task_1452796282942_0001_1_01_000000, scheduledTime=1452796283210, launchTime=1452796283210
2016-01-14 18:31:23,211 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282942_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:23,211 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9]
2016-01-14 18:31:23,211 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:23,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:23,211 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: F368293C2AAF42E18F0D597A146C4DE9, VertexParallelism: 1, TaskAttemptID:attempt_1452796282942_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B9CFE9BD304D4140AF7211026E582774, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=66B413E1CE85475EB0382F98FF4F2149, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:23,212 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282942_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:23,212 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:23,212 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796282942_0001_00_000002
2016-01-14 18:31:23,213 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:23,213 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796282942_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:23,213 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796282942_0001_00_000001: 13 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:23,213 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796282942_0001_00_000001, Exiting
2016-01-14 18:31:23,213 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796282942_0001_00_000001
2016-01-14 18:31:23,213 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796282942_0001_00_000001
2016-01-14 18:31:23,215 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796282942_0001_00_000002
2016-01-14 18:31:23,216 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796282942_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:23,216 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796282942_0001_00_000001 completed successfully
2016-01-14 18:31:23,217 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:23,216 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:23,219 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:23,220 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:23,220 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796282942_0001_00_000002, launchTime=1452796283220
2016-01-14 18:31:23,221 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796282942_0001_00_000002
2016-01-14 18:31:23,221 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796282942_0001_1_01_000000_0] to container: [container_1452796282942_0001_00_000002]
2016-01-14 18:31:23,221 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:23,222 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796282942_0001_1_01_000000_0] started. Is using containerId: [container_1452796282942_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:23,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F368293C2AAF42E18F0D597A146C4DE9, taskAttemptId=attempt_1452796282942_0001_1_01_000000_0, startTime=1452796283222, containerId=container_1452796282942_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796282942_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:23,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:23,222 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796282942_0001_00_000002 given task: attempt_1452796282942_0001_1_01_000000_0
2016-01-14 18:31:23,222 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282942_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:23,223 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282942_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:23,223 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796282942_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796282942_0001_1_01_000000_0
2016-01-14 18:31:23 Starting to run new task attempt: attempt_1452796282942_0001_1_01_000000_0
2016-01-14 18:31:23,223 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796282942_0001_1_01_000000_0
2016-01-14 18:31:23,223 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:23,224 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:23,224 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:23,224 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:23,224 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:23,225 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:23,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E] every/(1/1) ...latform/-1_VALUES_RESULTS, VertexName: F368293C2AAF42E18F0D597A146C4DE9, VertexParallelism: 1, TaskAttemptID:attempt_1452796282942_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B9CFE9BD304D4140AF7211026E582774, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=66B413E1CE85475EB0382F98FF4F2149, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:23,225 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:23,226 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796282942_0001_1_01_000000_0
2016-01-14 18:31:23,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:23,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,226 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796282942_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:23,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=66B413E1CE85475EB0382F98FF4F2149, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:23,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:23,228 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 66B413E1CE85475EB0382F98FF4F2149
2016-01-14 18:31:23,228 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:23,228 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 66B413E1CE85475EB0382F98FF4F2149
2016-01-14 18:31:23,230 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:23,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B9CFE9BD304D4140AF7211026E582774, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:23,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:23,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B9CFE9BD304D4140AF7211026E582774
2016-01-14 18:31:23,236 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:23,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 66B413E1CE85475EB0382F98FF4F2149
2016-01-14 18:31:23,237 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:23,238 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:23,238 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B9CFE9BD304D4140AF7211026E582774
2016-01-14 18:31:23,238 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F368293C2AAF42E18F0D597A146C4DE9, ordinal: 1
2016-01-14 18:31:23,239 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F368293C2AAF42E18F0D597A146C4DE9, mem on start (mb), free: 162, total: 369, max: 672
2016-01-14 18:31:23,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:23,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:23,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B9CFE9BD304D4140AF7211026E582774
2016-01-14 18:31:23,240 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:23,240 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:23,240 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:23,241 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:23,241 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:23,241 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:23,241 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:23,242 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 66B413E1CE85475EB0382F98FF4F2149, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:23,242 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B9CFE9BD304D4140AF7211026E582774, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:23,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:23,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B9CFE9BD304D4140AF7211026E582774 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:23,242 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B9CFE9BD304D4140AF7211026E582774
2016-01-14 18:31:23,243 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:23,244 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:23,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:23,244 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B9CFE9BD304D4140AF7211026E582774 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:23,244 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:23,244 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:23,244 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:23,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:23,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B9CFE9BD304D4140AF7211026E582774
2016-01-14 18:31:23,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:23,246 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796282942_0001_1_01_000000_0
2016-01-14 18:31:23,249 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:23,249 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 8021D5BC24344B309EFA50E0028DCAAC
2016-01-14 18:31:23,249 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS"], id: 66B413E1CE85475EB0382F98FF4F2149
2016-01-14 18:31:23,250 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_RESULTS"]
2016-01-14 18:31:23,250 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796282942_0001_1_00_000000_0_10069, runDuration: 43012, ]
2016-01-14 18:31:23,250 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 8021D5BC24344B309EFA50E0028DCAAC, for 1 inputs
2016-01-14 18:31:23,252 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796282942_0001_1_00_000000_0_10069] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796283252, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:23,252 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B9CFE9BD304D4140AF7211026E582774
2016-01-14 18:31:23,252 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:23,252 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B9CFE9BD304D4140AF7211026E582774] #12 in 1ms
2016-01-14 18:31:23,253 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B9CFE9BD304D4140AF7211026E582774] #12, status:false, isInterrupted:false
2016-01-14 18:31:23,253 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:23,253 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:23,254 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:23,254 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:23,254 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:23,254 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F368293C2AAF42E18F0D597A146C4DE9, all 1 inputs ready in: 00:00:00.004
2016-01-14 18:31:23,255 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B9CFE9BD304D4140AF7211026E582774
2016-01-14 18:31:23,255 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@795fffbf; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@d8f92da
2016-01-14 18:31:23,255 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:23,256 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:23,256 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:23,257 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:23,257 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:23,257 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796282942_0001_1_01_000000_0
2016-01-14 18:31:23,257 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796282942_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:23,258 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796282942_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:23,262 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F368293C2AAF42E18F0D597A146C4DE9, taskAttemptId=attempt_1452796282942_0001_1_01_000000_0, startTime=1452796283222, finishTime=1452796283262, timeTaken=40, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796283254, Process_Duration=4, Process_End_Time=1452796283258, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:23,262 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282942_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:23,263 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796282942_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:23,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:TASK_FINISHED]: vertexName=F368293C2AAF42E18F0D597A146C4DE9, taskId=task_1452796282942_0001_1_01_000000, startTime=1452796283222, finishTime=1452796283263, timeTaken=41, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:23,263 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796282942_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:23,264 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] : 1
2016-01-14 18:31:23,264 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] because task failed: task_1452796282942_0001_1_01_000000
2016-01-14 18:31:23,264 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:23,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:23,264 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:23,265 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:23,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:23,265 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9]
2016-01-14 18:31:23,267 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=66B413E1CE85475EB0382F98FF4F2149, vertexId=vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9]
2016-01-14 18:31:23,269 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:23,269 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:VERTEX_FINISHED]: vertexName=F368293C2AAF42E18F0D597A146C4DE9, vertexId=vertex_1452796282942_0001_1_01, initRequestedTime=1452796283113, initedTime=1452796283121, startRequestedTime=1452796283131, startedTime=1452796283131, finishTime=1452796283268, timeTaken=137, status=FAILED, diagnostics=Task failed, taskId=task_1452796282942_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796283222, firstTasksToStart=[ task_1452796282942_0001_1_01_000000 ], lastTaskFinishTime=1452796283262, lastTasksToFinish=[ task_1452796282942_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:23,269 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:23,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796283254, Process_Duration=4, Process_End_Time=1452796283258, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:23,275 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F368293C2AAF42E18F0D597A146C4DE9, mem on close (mb), free: 156, total: 369, max: 672
2016-01-14 18:31:23,275 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F368293C2AAF42E18F0D597A146C4DE9, index=1
2016-01-14 18:31:23,276 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796282942_0001_00_000002, stoppedTime=1452796283276, exitStatus=0
2016-01-14 18:31:23,276 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:23,276 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796282942_0001_1_01_000000_0 failed: true container: container_1452796282942_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:23,276 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:23,276 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:23,276 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796282942_0001_1
2016-01-14 18:31:23,276 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 66B413E1CE85475EB0382F98FF4F2149 for vertex: vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9]
2016-01-14 18:31:23,277 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796282942_0001_1_00 [B9CFE9BD304D4140AF7211026E582774]
2016-01-14 18:31:23,278 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:23,279 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796282942_0001_1, queueSize=0
2016-01-14 18:31:23,280 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B9CFE9BD304D4140AF7211026E582774
2016-01-14 18:31:23,280 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796282942_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796282942_0001_1, startTime=1452796283111, finishTime=1452796283277, timeTaken=166, status=FAILED, diagnostics=Vertex failed, vertexName=F368293C2AAF42E18F0D597A146C4DE9, vertexId=vertex_1452796282942_0001_1_01, diagnostics=[Task failed, taskId=task_1452796282942_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796282942_0001_1_01 [F368293C2AAF42E18F0D597A146C4DE9] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796283188, Process_Duration=4, Process_End_Time=1452796283192, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:23,280 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796282942_0001_1 finished with state: FAILED
2016-01-14 18:31:23,280 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796282942_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:23 Completed Dag: dag_1452796282942_0001_1
2016-01-14 18:31:23,281 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796282942_0001_1, dagState=FAILED
2016-01-14 18:31:23,281 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796282942_0001_1_post
2016-01-14 18:31:23,281 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:23,281 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:23,281 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796282942_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:23,282 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:23,282 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:23,282 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:23,282 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:23,282 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:23,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F368293C2AAF42E18F0D597A146C4DE9, sourceVertex=B9CFE9BD304D4140AF7211026E582774
2016-01-14 18:31:23,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F368293C2AAF42E18F0D597A146C4DE9, sourceVertex=66B413E1CE85475EB0382F98FF4F2149
2016-01-14 18:31:23,283 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796282942_0001_00_000002
2016-01-14 18:31:23,283 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796282942_0001_00_000002
2016-01-14 18:31:23,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:23,284 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:23,284 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796282942_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:23,284 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796282942_0001_00_000002 completed but with errors
2016-01-14 18:31:23,286 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796282942_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:23,286 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796282942_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:23,286 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796282942_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:23,293 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/-1_VALUES_RESULTS, applicationId=application_1452796282942_0001
2016-01-14 18:31:23,295 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:23,295 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:23,296 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:23,296 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:23,296 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:23,297 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:23,297 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:23,297 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:23,298 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:23,298 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:23,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:23,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:23,300 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:23,304 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F798E02376D745819F10A9314166A37C/E080AFBDCF654FD786E1EBC7D613C99E/.tez/application_1452796282942_0001
2016-01-14 18:31:23,304 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:23,304 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...latform/-1_VALUES_RESULTS
2016-01-14 18:31:23,305 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:23,305 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.383
2016-01-14 18:31:23,306 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_RESULTS
2016-01-14 18:31:23,333 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:23,333 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:23,375 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 18:31:23,388 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.028
2016-01-14 18:31:23,388 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:23,388 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:23,389 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:23,408 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:23,415 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL
2016-01-14 18:31:23,471 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:23,471 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:23,471 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL"]
2016-01-14 18:31:23,472 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:23,473 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:23,473 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:23,474 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 18:31:23,474 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:23,487 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:23,491 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:23,492 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:23,492 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:23,501 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22/.tez/application_1452796283491_0001 doesn't exist and is created
2016-01-14 18:31:23,515 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:23,515 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:23,516 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22/.tez/application_1452796283491_0001
2016-01-14 18:31:23,518 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796283491_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:23,530 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:23,531 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:23,533 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:23,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:23,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22/.tez/application_1452796283491_0001/localmode-log-dir
2016-01-14 18:31:23,534 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22/.tez/application_1452796283491_0001/localmode-log-dir/history.txt.appattempt_1452796283491_0001_000000, maxErrors=10
2016-01-14 18:31:23,534 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:23,535 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796283491_0001_000000, appSubmitTime=1452796283518, launchTime=1452796283518
2016-01-14 18:31:23,541 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:60423
2016-01-14 18:31:23,544 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:23,545 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:23,545 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:23,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:23,547 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:23,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796283491_0001_000000, startTime=1452796283550
2016-01-14 18:31:23,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:23,550 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:23,615 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:23,616 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:23,616 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1452796283491_0001, dagName=[E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 18:31:23,635 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 18:31:23,637 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796283491_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22/.tez/application_1452796283491_0001/localmode-log-dir/dag_1452796283491_0001_1.dot
2016-01-14 18:31:23,638 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22/.tez/application_1452796283491_0001/localmode-log-dir/dag_1452796283491_0001_1-tez-dag.pb.txt
2016-01-14 18:31:23,652 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796283491_0001_1
2016-01-14 18:31:23,652 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:23,653 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 18:31:23 Running Dag: dag_1452796283491_0001_1
2016-01-14 18:31:23,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796283491_0001_1, submitTime=1452796283635
2016-01-14 18:31:23,663 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:23,664 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CD2B5B69242A436FB755BA5A65546CC1
2016-01-14 18:31:23,664 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:23,665 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796283491_0001_1, initTime=1452796283663
2016-01-14 18:31:23,665 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:23,665 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796283491_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:23,667 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:DAG_STARTED]: dagID=dag_1452796283491_0001_1, startTime=1452796283666
2016-01-14 18:31:23,667 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:23,667 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:23,668 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796283491_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:23,668 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1452796283491_0001, dagName=[E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22] every/(1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 18:31:23,668 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BBE5137BB46140DD8561675E1EA1F0A6 : {D46331D87A104C2EBE893B25841107B1={InputName=D46331D87A104C2EBE893B25841107B1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:23,668 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D46331D87A104C2EBE893B25841107B1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:23,668 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6]
2016-01-14 18:31:23,668 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6]
2016-01-14 18:31:23,668 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6]
2016-01-14 18:31:23,669 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6]. Starting root input initializers: 1
2016-01-14 18:31:23,669 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:23,669 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D46331D87A104C2EBE893B25841107B1 on vertex vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6]
2016-01-14 18:31:23,669 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1]
2016-01-14 18:31:23,671 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:23,671 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D46331D87A104C2EBE893B25841107B1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:23,672 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:23,672 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:23,673 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:23,673 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1]
2016-01-14 18:31:23,673 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1]
2016-01-14 18:31:23,673 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1]
2016-01-14 18:31:23,674 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:23,674 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D46331D87A104C2EBE893B25841107B1 on vertex vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6]
2016-01-14 18:31:23,674 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=35B0B8A3BC414BF18FE2DBC041BA56E4, vertexId=vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:23,674 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=35B0B8A3BC414BF18FE2DBC041BA56E4, vertexId=vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1]
2016-01-14 18:31:23,676 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=35B0B8A3BC414BF18FE2DBC041BA56E4, vertexName=CD2B5B69242A436FB755BA5A65546CC1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:23,676 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=35B0B8A3BC414BF18FE2DBC041BA56E4, vertexId=vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1]
2016-01-14 18:31:23,677 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:23,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CD2B5B69242A436FB755BA5A65546CC1, vertexId=vertex_1452796283491_0001_1_01, initRequestedTime=1452796283669, initedTime=1452796283677, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:23,677 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:23,678 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:23,678 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D46331D87A104C2EBE893B25841107B1=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:23,678 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6] parallelism set to 1
2016-01-14 18:31:23,678 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:23,679 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796283491_0001_1_00, eventCount=1
2016-01-14 18:31:23,679 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:23,679 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BBE5137BB46140DD8561675E1EA1F0A6, vertexId=vertex_1452796283491_0001_1_00, initRequestedTime=1452796283668, initedTime=1452796283679, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:23,680 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6] with distanceFromRoot: 0
2016-01-14 18:31:23,680 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:23,680 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BBE5137BB46140DD8561675E1EA1F0A6
2016-01-14 18:31:23,680 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6]
2016-01-14 18:31:23,681 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:23,681 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796283491_0001_1_00, startRequestedTime=1452796283680, startedTime=1452796283680
2016-01-14 18:31:23,681 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:23,682 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:23,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:TASK_STARTED]: vertexName=BBE5137BB46140DD8561675E1EA1F0A6, taskId=task_1452796283491_0001_1_00_000000, scheduledTime=1452796283681, launchTime=1452796283681
2016-01-14 18:31:23,682 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796283491_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:23,682 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796283491_0001_1_00 for vertex: vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] numStartedSources: 1 numSources: 1
2016-01-14 18:31:23,682 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] with distanceFromRoot: 1
2016-01-14 18:31:23,683 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6]
2016-01-14 18:31:23,683 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BBE5137BB46140DD8561675E1EA1F0A6 in vertex: CD2B5B69242A436FB755BA5A65546CC1
2016-01-14 18:31:23,683 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CD2B5B69242A436FB755BA5A65546CC1 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:23,684 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:23,684 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796283491_0001_1_01, startRequestedTime=1452796283683, startedTime=1452796283683
2016-01-14 18:31:23,684 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:23,684 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: BBE5137BB46140DD8561675E1EA1F0A6, VertexParallelism: 1, TaskAttemptID:attempt_1452796283491_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D46331D87A104C2EBE893B25841107B1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CD2B5B69242A436FB755BA5A65546CC1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:23,685 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796283491_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:23,685 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:23,686 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:23,686 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:23,686 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:23,686 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796283491_0001_00_000001
2016-01-14 18:31:23,687 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:23,688 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796283491_0001_00_000001
2016-01-14 18:31:23,688 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:23,689 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:23,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:23,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796283491_0001_00_000001, launchTime=1452796283689
2016-01-14 18:31:23,690 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796283491_0001_00_000001
2016-01-14 18:31:23,691 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796283491_0001_1_00_000000_0] to container: [container_1452796283491_0001_00_000001]
2016-01-14 18:31:23,691 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:23,692 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796283491_0001_1_00_000000_0] started. Is using containerId: [container_1452796283491_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:23,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BBE5137BB46140DD8561675E1EA1F0A6, taskAttemptId=attempt_1452796283491_0001_1_00_000000_0, startTime=1452796283691, containerId=container_1452796283491_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796283491_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:23,692 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796283491_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:23,692 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796283491_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:23,692 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796283491_0001_00_000001 given task: attempt_1452796283491_0001_1_00_000000_0
2016-01-14 18:31:23,692 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796283491_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796283491_0001_1_00_000000_0
2016-01-14 18:31:23,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:23 Starting to run new task attempt: attempt_1452796283491_0001_1_00_000000_0
2016-01-14 18:31:23,693 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796283491_0001_1_00_000000_0
2016-01-14 18:31:23,693 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:23,693 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:23,693 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:23,693 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796283491_0001, with dag id: dag_1452796283491_0001_1
2016-01-14 18:31:23,693 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:23,696 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:23,696 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:23,697 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: BBE5137BB46140DD8561675E1EA1F0A6, VertexParallelism: 1, TaskAttemptID:attempt_1452796283491_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D46331D87A104C2EBE893B25841107B1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CD2B5B69242A436FB755BA5A65546CC1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:23,697 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:23,698 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796283491_0001_1_00_000000_0
2016-01-14 18:31:23,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:23,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D46331D87A104C2EBE893B25841107B1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:23,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:23,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D46331D87A104C2EBE893B25841107B1
2016-01-14 18:31:23,699 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D46331D87A104C2EBE893B25841107B1
2016-01-14 18:31:23,703 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CD2B5B69242A436FB755BA5A65546CC1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:23,703 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:23,703 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:23,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CD2B5B69242A436FB755BA5A65546CC1
2016-01-14 18:31:23,703 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:23,705 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D46331D87A104C2EBE893B25841107B1
2016-01-14 18:31:23,705 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D46331D87A104C2EBE893B25841107B1
2016-01-14 18:31:23,705 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796283491_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:23,706 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:23,706 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CD2B5B69242A436FB755BA5A65546CC1
2016-01-14 18:31:23,707 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CD2B5B69242A436FB755BA5A65546CC1
2016-01-14 18:31:23,710 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BBE5137BB46140DD8561675E1EA1F0A6, ordinal: 0
2016-01-14 18:31:23,710 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BBE5137BB46140DD8561675E1EA1F0A6, mem on start (mb), free: 278, total: 369, max: 672
2016-01-14 18:31:23,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:23,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:23,711 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:23,711 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:23,711 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:23,711 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:23,711 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:23,711 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:23,712 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:23,712 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D46331D87A104C2EBE893B25841107B1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:23,712 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CD2B5B69242A436FB755BA5A65546CC1, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:23,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:23,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D46331D87A104C2EBE893B25841107B1 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:23,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:23,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:23,712 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D46331D87A104C2EBE893B25841107B1
2016-01-14 18:31:23,713 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:23,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D46331D87A104C2EBE893B25841107B1
2016-01-14 18:31:23,713 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:23,713 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796283491_0001_1_00_000000_0
2016-01-14 18:31:23,714 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:23,714 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3b0219c6
2016-01-14 18:31:23,715 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3b0219c6
2016-01-14 18:31:23,716 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:23,716 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:23,717 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:23,718 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D46331D87A104C2EBE893B25841107B1
2016-01-14 18:31:23,718 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: BB7A965A79A4425382B6283FCE21D3EB
2016-01-14 18:31:23,719 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] BB7A965A79A4425382B6283FCE21D3EB
2016-01-14 18:31:23,719 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@c086931; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7c321622; comparator=cascading.tuple.hadoop.util.TupleComparator@3c4b51e1; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:23,720 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:23,720 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:23,720 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:23,733 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:23,734 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:23,734 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:23,734 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:23,734 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:23,735 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:23,735 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:23,735 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BBE5137BB46140DD8561675E1EA1F0A6, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:23,738 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796283491_0001_1_00_000000_0
2016-01-14 18:31:23,738 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BBE5137BB46140DD8561675E1EA1F0A6, mem on close (mb), free: 175, total: 369, max: 672
2016-01-14 18:31:23,738 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:23,738 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:23,739 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:23,739 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:23,740 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:23 Completed running task attempt: attempt_1452796283491_0001_1_00_000000_0
2016-01-14 18:31:23,743 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796283491_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:23,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796283491_0001_1_00, eventCount=1
2016-01-14 18:31:23,744 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:23,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BBE5137BB46140DD8561675E1EA1F0A6, taskAttemptId=attempt_1452796283491_0001_1_00_000000_0, startTime=1452796283691, finishTime=1452796283744, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796283736, Process_Duration=2, Process_End_Time=1452796283738, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:23,745 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796283491_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:23,745 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796283491_0001_1_00_000000_0
2016-01-14 18:31:23,745 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:23,746 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:TASK_FINISHED]: vertexName=BBE5137BB46140DD8561675E1EA1F0A6, taskId=task_1452796283491_0001_1_00_000000, startTime=1452796283691, finishTime=1452796283745, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452796283491_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796283736, Process_Duration=2, Process_End_Time=1452796283738, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:23,746 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796283491_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:23,746 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:23,746 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6] : 1
2016-01-14 18:31:23,746 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6]
2016-01-14 18:31:23,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796283736, Process_Duration=2, Process_End_Time=1452796283738, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:23,747 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:23,747 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:23,748 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796283491_0001_00_000001
2016-01-14 18:31:23,748 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:VERTEX_FINISHED]: vertexName=BBE5137BB46140DD8561675E1EA1F0A6, vertexId=vertex_1452796283491_0001_1_00, initRequestedTime=1452796283668, initedTime=1452796283679, startRequestedTime=1452796283680, startedTime=1452796283680, finishTime=1452796283746, timeTaken=66, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796283736, Process_Duration=2, Process_End_Time=1452796283738, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796283691, firstTasksToStart=[ task_1452796283491_0001_1_00_000000 ], lastTaskFinishTime=1452796283744, lastTasksToFinish=[ task_1452796283491_0001_1_00_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796283491_0001_1_00_000000 ], longestDurationTasks=[ task_1452796283491_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:23,748 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:23,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:23,748 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:23,749 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796283491_0001_00_000001, stoppedTime=1452796283749, exitStatus=0
2016-01-14 18:31:23,749 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:23,749 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:23,749 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] attempt: attempt_1452796283491_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:23,750 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CD2B5B69242A436FB755BA5A65546CC1
2016-01-14 18:31:23,750 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1]
2016-01-14 18:31:23,750 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:23,750 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796283491_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:23,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:TASK_STARTED]: vertexName=CD2B5B69242A436FB755BA5A65546CC1, taskId=task_1452796283491_0001_1_01_000000, scheduledTime=1452796283751, launchTime=1452796283751
2016-01-14 18:31:23,751 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796283491_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:23,751 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:23,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:23,751 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1]
2016-01-14 18:31:23,752 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: CD2B5B69242A436FB755BA5A65546CC1, VertexParallelism: 1, TaskAttemptID:attempt_1452796283491_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BBE5137BB46140DD8561675E1EA1F0A6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=35B0B8A3BC414BF18FE2DBC041BA56E4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:23,752 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796283491_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:23,752 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:23,753 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796283491_0001_00_000002
2016-01-14 18:31:23,753 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:23,754 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796283491_0001_00_000002
2016-01-14 18:31:23,754 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:23,755 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:23,755 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:23,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796283491_0001_00_000002, launchTime=1452796283755
2016-01-14 18:31:23,756 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796283491_0001_00_000002
2016-01-14 18:31:23,757 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796283491_0001_1_01_000000_0] to container: [container_1452796283491_0001_00_000002]
2016-01-14 18:31:23,758 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:23,758 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796283491_0001_00_000002 given task: attempt_1452796283491_0001_1_01_000000_0
2016-01-14 18:31:23,758 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796283491_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796283491_0001_1_01_000000_0
2016-01-14 18:31:23,758 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796283491_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:23,759 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796283491_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:23,758 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796283491_0001_1_01_000000_0] started. Is using containerId: [container_1452796283491_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:23,759 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796283491_0001_00_000001, Exiting
2016-01-14 18:31:23,759 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796283491_0001_00_000001
2016-01-14 18:31:23 Starting to run new task attempt: attempt_1452796283491_0001_1_01_000000_0
2016-01-14 18:31:23,759 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796283491_0001_00_000001
2016-01-14 18:31:23,759 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:23,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CD2B5B69242A436FB755BA5A65546CC1, taskAttemptId=attempt_1452796283491_0001_1_01_000000_0, startTime=1452796283758, containerId=container_1452796283491_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796283491_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:23,759 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796283491_0001_1_01_000000_0
2016-01-14 18:31:23,760 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:23,760 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:23,760 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:23,760 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:23,760 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:23,760 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:23,760 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22] every/(1/1) ...blyplatform/-1_VALUES_ALL, VertexName: CD2B5B69242A436FB755BA5A65546CC1, VertexParallelism: 1, TaskAttemptID:attempt_1452796283491_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BBE5137BB46140DD8561675E1EA1F0A6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=35B0B8A3BC414BF18FE2DBC041BA56E4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:23,761 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:23,760 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796283491_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:23,761 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796283491_0001_1_01_000000_0
2016-01-14 18:31:23,761 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796283491_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:23,762 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796283491_0001_00_000001 completed successfully
2016-01-14 18:31:23,761 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796283491_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:23,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:23,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,762 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:23,763 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796283491_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:23,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=35B0B8A3BC414BF18FE2DBC041BA56E4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:23,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:23,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 35B0B8A3BC414BF18FE2DBC041BA56E4
2016-01-14 18:31:23,764 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:23,765 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 35B0B8A3BC414BF18FE2DBC041BA56E4
2016-01-14 18:31:23,766 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:23,768 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BBE5137BB46140DD8561675E1EA1F0A6, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:23,769 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:23,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BBE5137BB46140DD8561675E1EA1F0A6
2016-01-14 18:31:23,772 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:23,772 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:23,772 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BBE5137BB46140DD8561675E1EA1F0A6
2016-01-14 18:31:23,773 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BBE5137BB46140DD8561675E1EA1F0A6
2016-01-14 18:31:23,775 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CD2B5B69242A436FB755BA5A65546CC1, ordinal: 1
2016-01-14 18:31:23,775 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CD2B5B69242A436FB755BA5A65546CC1, mem on start (mb), free: 168, total: 369, max: 672
2016-01-14 18:31:23,775 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:23,775 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:23,775 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:23,776 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:23,776 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 35B0B8A3BC414BF18FE2DBC041BA56E4
2016-01-14 18:31:23,776 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:23,777 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:23,777 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:23,777 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:23,777 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:23,778 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:23,778 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:23,778 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 35B0B8A3BC414BF18FE2DBC041BA56E4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:23,778 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BBE5137BB46140DD8561675E1EA1F0A6, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:23,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:23,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BBE5137BB46140DD8561675E1EA1F0A6 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:23,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BBE5137BB46140DD8561675E1EA1F0A6
2016-01-14 18:31:23,780 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:23,780 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:23,780 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BBE5137BB46140DD8561675E1EA1F0A6 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:23,780 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:23,780 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:23,781 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:23,781 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:23,781 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:23,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BBE5137BB46140DD8561675E1EA1F0A6
2016-01-14 18:31:23,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:23,782 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796283491_0001_1_01_000000_0
2016-01-14 18:31:23,785 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796283491_0001_1_00_000000_0_10075, runDuration: 37753, ]
2016-01-14 18:31:23,785 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:23,786 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: BB7A965A79A4425382B6283FCE21D3EB
2016-01-14 18:31:23,786 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL"], id: 35B0B8A3BC414BF18FE2DBC041BA56E4
2016-01-14 18:31:23,786 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_ALL"]
2016-01-14 18:31:23,786 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] BB7A965A79A4425382B6283FCE21D3EB, for 1 inputs
2016-01-14 18:31:23,787 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796283491_0001_1_00_000000_0_10075] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796283787, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:23,787 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BBE5137BB46140DD8561675E1EA1F0A6
2016-01-14 18:31:23,787 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:23,787 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BBE5137BB46140DD8561675E1EA1F0A6] #13 in 2ms
2016-01-14 18:31:23,788 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BBE5137BB46140DD8561675E1EA1F0A6] #13, status:false, isInterrupted:false
2016-01-14 18:31:23,789 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:23,789 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:23,789 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:23,789 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:23,790 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:23,790 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BBE5137BB46140DD8561675E1EA1F0A6
2016-01-14 18:31:23,790 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CD2B5B69242A436FB755BA5A65546CC1, all 1 inputs ready in: 00:00:00.004
2016-01-14 18:31:23,790 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:23,792 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2cdd03e5; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@622ac665
2016-01-14 18:31:23,793 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:23,793 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:23,794 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:23,794 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:23,794 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796283491_0001_1_01_000000_0
2016-01-14 18:31:23,794 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796283491_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:23,795 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796283491_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:23,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:23,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CD2B5B69242A436FB755BA5A65546CC1, taskAttemptId=attempt_1452796283491_0001_1_01_000000_0, startTime=1452796283758, finishTime=1452796283796, timeTaken=38, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796283790, Process_Duration=5, Process_End_Time=1452796283795, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:23,797 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796283491_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:23,797 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796283491_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:23,798 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:23,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:TASK_FINISHED]: vertexName=CD2B5B69242A436FB755BA5A65546CC1, taskId=task_1452796283491_0001_1_01_000000, startTime=1452796283758, finishTime=1452796283797, timeTaken=39, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:23,798 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796283491_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:23,798 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] : 1
2016-01-14 18:31:23,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796283790, Process_Duration=5, Process_End_Time=1452796283795, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:23,798 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:23,798 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] because task failed: task_1452796283491_0001_1_01_000000
2016-01-14 18:31:23,799 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:23,799 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CD2B5B69242A436FB755BA5A65546CC1, mem on close (mb), free: 163, total: 369, max: 672
2016-01-14 18:31:23,799 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:23,799 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1]
2016-01-14 18:31:23,799 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=35B0B8A3BC414BF18FE2DBC041BA56E4, vertexId=vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1]
2016-01-14 18:31:23,801 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:23,801 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:VERTEX_FINISHED]: vertexName=CD2B5B69242A436FB755BA5A65546CC1, vertexId=vertex_1452796283491_0001_1_01, initRequestedTime=1452796283669, initedTime=1452796283677, startRequestedTime=1452796283683, startedTime=1452796283683, finishTime=1452796283800, timeTaken=117, status=FAILED, diagnostics=Task failed, taskId=task_1452796283491_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796283758, firstTasksToStart=[ task_1452796283491_0001_1_01_000000 ], lastTaskFinishTime=1452796283796, lastTasksToFinish=[ task_1452796283491_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:23,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=CD2B5B69242A436FB755BA5A65546CC1, index=1
2016-01-14 18:31:23,801 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:23,801 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BBE5137BB46140DD8561675E1EA1F0A6
2016-01-14 18:31:23,802 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:23,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796283491_0001_00_000002, stoppedTime=1452796283802, exitStatus=0
2016-01-14 18:31:23,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:23,802 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:23,802 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:23,802 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:23,803 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:23,803 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:23,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=CD2B5B69242A436FB755BA5A65546CC1, sourceVertex=BBE5137BB46140DD8561675E1EA1F0A6
2016-01-14 18:31:23,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=CD2B5B69242A436FB755BA5A65546CC1, sourceVertex=35B0B8A3BC414BF18FE2DBC041BA56E4
2016-01-14 18:31:23,802 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796283491_0001_1_01_000000_0 failed: true container: container_1452796283491_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:23,803 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:23,803 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:23,803 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796283491_0001_1
2016-01-14 18:31:23,804 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 35B0B8A3BC414BF18FE2DBC041BA56E4 for vertex: vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1]
2016-01-14 18:31:23,804 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796283491_0001_1_00 [BBE5137BB46140DD8561675E1EA1F0A6]
2016-01-14 18:31:23,806 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796283491_0001_1, queueSize=0
2016-01-14 18:31:23,806 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:23,806 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796283491_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796283491_0001_1, startTime=1452796283666, finishTime=1452796283804, timeTaken=138, status=FAILED, diagnostics=Vertex failed, vertexName=CD2B5B69242A436FB755BA5A65546CC1, vertexId=vertex_1452796283491_0001_1_01, diagnostics=[Task failed, taskId=task_1452796283491_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796283491_0001_1_01 [CD2B5B69242A436FB755BA5A65546CC1] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796283736, Process_Duration=2, Process_End_Time=1452796283738, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:23,806 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796283491_0001_1 finished with state: FAILED
2016-01-14 18:31:23,807 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796283491_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:23,807 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796283491_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:23,807 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796283491_0001_00_000002
2016-01-14 18:31:23,807 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796283491_0001_00_000002
2016-01-14 18:31:23,807 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796283491_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:23,807 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796283491_0001_00_000002 completed but with errors
2016-01-14 18:31:23 Completed Dag: dag_1452796283491_0001_1
2016-01-14 18:31:23,808 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796283491_0001_1, dagState=FAILED
2016-01-14 18:31:23,808 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796283491_0001_1_post
2016-01-14 18:31:23,808 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:23,808 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:23,808 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:23,808 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796283491_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:23,808 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796283491_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:23,808 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796283491_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:23,816 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/-1_VALUES_ALL, applicationId=application_1452796283491_0001
2016-01-14 18:31:23,818 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:23,818 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:23,819 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:23,819 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:23,819 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:23,819 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:23,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:23,820 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:23,820 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:23,820 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:23,820 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:23,820 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:23,821 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:23,824 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E4F94AA4DD8D474184878B2B16F551BB/CE6F908C858F4109B3252F6A9A6ADE22/.tez/application_1452796283491_0001
2016-01-14 18:31:23,824 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:23,825 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...blyplatform/-1_VALUES_ALL
2016-01-14 18:31:23,825 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:23,825 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.354
2016-01-14 18:31:23,826 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_ALL
2016-01-14 18:31:23,859 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:23,860 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:23,913 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:23,913 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:23,915 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 18:31:23,926 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.026
2016-01-14 18:31:23,927 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:23,927 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:23,927 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:23,942 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:23,948 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE
2016-01-14 18:31:24,004 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:24,004 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:24,004 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE"]
2016-01-14 18:31:24,004 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:24,004 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:24,005 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:24,006 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 18:31:24,007 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:24,022 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:24,026 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:24,026 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:24,026 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:24,036 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C/.tez/application_1452796284026_0001 doesn't exist and is created
2016-01-14 18:31:24,050 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:24,050 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:24,051 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C/.tez/application_1452796284026_0001
2016-01-14 18:31:24,053 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796284026_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:24,065 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:24,065 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:24,067 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:24,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:24,067 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C/.tez/application_1452796284026_0001/localmode-log-dir
2016-01-14 18:31:24,068 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C/.tez/application_1452796284026_0001/localmode-log-dir/history.txt.appattempt_1452796284026_0001_000000, maxErrors=10
2016-01-14 18:31:24,068 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:24,068 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796284026_0001_000000, appSubmitTime=1452796284053, launchTime=1452796284053
2016-01-14 18:31:24,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:24,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:24,076 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:24,076 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:24,079 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:47196
2016-01-14 18:31:24,081 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:24,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796284026_0001_000000, startTime=1452796284083
2016-01-14 18:31:24,083 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:24,083 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:24,151 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:24,151 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:24,151 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1452796284026_0001, dagName=[2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 18:31:24,169 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 18:31:24,171 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796284026_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C/.tez/application_1452796284026_0001/localmode-log-dir/dag_1452796284026_0001_1.dot
2016-01-14 18:31:24,171 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C/.tez/application_1452796284026_0001/localmode-log-dir/dag_1452796284026_0001_1-tez-dag.pb.txt
2016-01-14 18:31:24,180 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796284026_0001_1
2016-01-14 18:31:24,180 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:24,180 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 18:31:24 Running Dag: dag_1452796284026_0001_1
2016-01-14 18:31:24,191 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:24,191 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796284026_0001_1, submitTime=1452796284169
2016-01-14 18:31:24,192 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A78EE660D55044F5AA1DEE245FCE0705
2016-01-14 18:31:24,192 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:24,193 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:24,193 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796284026_0001_1, initTime=1452796284192
2016-01-14 18:31:24,193 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796284026_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:24,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:DAG_STARTED]: dagID=dag_1452796284026_0001_1, startTime=1452796284194
2016-01-14 18:31:24,194 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:24,194 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:24,194 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796284026_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:24,194 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DDB94BB5FAC64F289A166CD9E42CDE71 : {7A30A3FAC84B4241A782859E47CF1614={InputName=7A30A3FAC84B4241A782859E47CF1614}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:24,194 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 7A30A3FAC84B4241A782859E47CF1614, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:24,194 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71]
2016-01-14 18:31:24,195 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71]
2016-01-14 18:31:24,195 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71]
2016-01-14 18:31:24,195 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71]. Starting root input initializers: 1
2016-01-14 18:31:24,195 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:24,196 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705]
2016-01-14 18:31:24,196 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 7A30A3FAC84B4241A782859E47CF1614 on vertex vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71]
2016-01-14 18:31:24,197 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:24,197 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 7A30A3FAC84B4241A782859E47CF1614 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:24,198 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:24,198 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:24,199 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:24,199 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705]
2016-01-14 18:31:24,199 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:24,200 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705]
2016-01-14 18:31:24,200 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 7A30A3FAC84B4241A782859E47CF1614 on vertex vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71]
2016-01-14 18:31:24,200 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705]
2016-01-14 18:31:24,200 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=74D6DA6EA6E64EC69DE66A08681AEC62, vertexId=vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:24,201 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1452796284026_0001, dagName=[2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C] every/(1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 18:31:24,203 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=74D6DA6EA6E64EC69DE66A08681AEC62, vertexId=vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705]
2016-01-14 18:31:24,204 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=74D6DA6EA6E64EC69DE66A08681AEC62, vertexName=A78EE660D55044F5AA1DEE245FCE0705, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:24,204 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=74D6DA6EA6E64EC69DE66A08681AEC62, vertexId=vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705]
2016-01-14 18:31:24,205 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:24,206 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A78EE660D55044F5AA1DEE245FCE0705, vertexId=vertex_1452796284026_0001_1_01, initRequestedTime=1452796284196, initedTime=1452796284205, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:24,206 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:24,207 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:24,207 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {7A30A3FAC84B4241A782859E47CF1614=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:24,208 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71] parallelism set to 1
2016-01-14 18:31:24,208 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:24,209 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796284026_0001_1_00, eventCount=1
2016-01-14 18:31:24,210 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:24,210 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DDB94BB5FAC64F289A166CD9E42CDE71, vertexId=vertex_1452796284026_0001_1_00, initRequestedTime=1452796284194, initedTime=1452796284209, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:24,211 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71] with distanceFromRoot: 0
2016-01-14 18:31:24,211 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:24,211 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DDB94BB5FAC64F289A166CD9E42CDE71
2016-01-14 18:31:24,212 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71]
2016-01-14 18:31:24,213 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:24,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796284026_0001_1_00, startRequestedTime=1452796284211, startedTime=1452796284211
2016-01-14 18:31:24,214 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:24,214 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:24,215 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:TASK_STARTED]: vertexName=DDB94BB5FAC64F289A166CD9E42CDE71, taskId=task_1452796284026_0001_1_00_000000, scheduledTime=1452796284214, launchTime=1452796284214
2016-01-14 18:31:24,216 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284026_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:24,216 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796284026_0001_1_00 for vertex: vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] numStartedSources: 1 numSources: 1
2016-01-14 18:31:24,216 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] with distanceFromRoot: 1
2016-01-14 18:31:24,217 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71]
2016-01-14 18:31:24,217 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: DDB94BB5FAC64F289A166CD9E42CDE71 in vertex: A78EE660D55044F5AA1DEE245FCE0705
2016-01-14 18:31:24,218 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A78EE660D55044F5AA1DEE245FCE0705 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:24,219 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:24,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796284026_0001_1_01, startRequestedTime=1452796284217, startedTime=1452796284217
2016-01-14 18:31:24,219 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:24,220 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: DDB94BB5FAC64F289A166CD9E42CDE71, VertexParallelism: 1, TaskAttemptID:attempt_1452796284026_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7A30A3FAC84B4241A782859E47CF1614, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A78EE660D55044F5AA1DEE245FCE0705, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:24,221 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284026_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:24,221 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:24,222 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:24,223 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:24,223 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:24,223 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796284026_0001, with dag id: dag_1452796284026_0001_1
2016-01-14 18:31:24,223 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796284026_0001_00_000001
2016-01-14 18:31:24,223 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:24,226 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796284026_0001_00_000001
2016-01-14 18:31:24,226 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:24,226 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:24,227 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:24,228 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796284026_0001_00_000001, launchTime=1452796284227
2016-01-14 18:31:24,229 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796284026_0001_00_000001
2016-01-14 18:31:24,230 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796284026_0001_1_00_000000_0] to container: [container_1452796284026_0001_00_000001]
2016-01-14 18:31:24,230 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:24,231 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796284026_0001_1_00_000000_0] started. Is using containerId: [container_1452796284026_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:24,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DDB94BB5FAC64F289A166CD9E42CDE71, taskAttemptId=attempt_1452796284026_0001_1_00_000000_0, startTime=1452796284230, containerId=container_1452796284026_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796284026_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:24,231 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284026_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:24,231 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284026_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:24,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:24,231 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796284026_0001_00_000001 given task: attempt_1452796284026_0001_1_00_000000_0
2016-01-14 18:31:24,231 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796284026_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796284026_0001_1_00_000000_0
2016-01-14 18:31:24 Starting to run new task attempt: attempt_1452796284026_0001_1_00_000000_0
2016-01-14 18:31:24,232 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796284026_0001_1_00_000000_0
2016-01-14 18:31:24,232 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:24,232 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:24,232 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:24,233 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:24,233 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:24,233 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:24,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: DDB94BB5FAC64F289A166CD9E42CDE71, VertexParallelism: 1, TaskAttemptID:attempt_1452796284026_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=7A30A3FAC84B4241A782859E47CF1614, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A78EE660D55044F5AA1DEE245FCE0705, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:24,233 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:24,234 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796284026_0001_1_00_000000_0
2016-01-14 18:31:24,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:24,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,235 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796284026_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:24,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=7A30A3FAC84B4241A782859E47CF1614, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:24,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A78EE660D55044F5AA1DEE245FCE0705, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:24,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:24,237 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A78EE660D55044F5AA1DEE245FCE0705
2016-01-14 18:31:24,239 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:24,239 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A78EE660D55044F5AA1DEE245FCE0705
2016-01-14 18:31:24,236 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:24,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 7A30A3FAC84B4241A782859E47CF1614
2016-01-14 18:31:24,240 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 7A30A3FAC84B4241A782859E47CF1614
2016-01-14 18:31:24,239 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A78EE660D55044F5AA1DEE245FCE0705
2016-01-14 18:31:24,239 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:24,245 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:24,246 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 7A30A3FAC84B4241A782859E47CF1614
2016-01-14 18:31:24,246 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 7A30A3FAC84B4241A782859E47CF1614
2016-01-14 18:31:24,247 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DDB94BB5FAC64F289A166CD9E42CDE71, ordinal: 0
2016-01-14 18:31:24,247 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DDB94BB5FAC64F289A166CD9E42CDE71, mem on start (mb), free: 283, total: 368, max: 672
2016-01-14 18:31:24,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:24,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:24,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:24,247 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:24,248 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:24,248 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:24,248 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:24,248 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:24,248 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:24,248 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A78EE660D55044F5AA1DEE245FCE0705, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:24,249 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 7A30A3FAC84B4241A782859E47CF1614, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:24,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:24,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 7A30A3FAC84B4241A782859E47CF1614 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:24,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:24,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:24,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 7A30A3FAC84B4241A782859E47CF1614
2016-01-14 18:31:24,250 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:24,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 7A30A3FAC84B4241A782859E47CF1614
2016-01-14 18:31:24,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:24,251 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796284026_0001_1_00_000000_0
2016-01-14 18:31:24,252 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:24,253 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@663c7162
2016-01-14 18:31:24,255 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:24,256 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 7A30A3FAC84B4241A782859E47CF1614
2016-01-14 18:31:24,256 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 964E8D10BA43412E8A3C4DFA3F11780A
2016-01-14 18:31:24,256 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 964E8D10BA43412E8A3C4DFA3F11780A
2016-01-14 18:31:24,257 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@424ad4c4; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@3ed6b1dc; comparator=cascading.tuple.hadoop.util.TupleComparator@13702bf; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:24,257 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:24,257 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:24,258 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:24,257 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@663c7162
2016-01-14 18:31:24,258 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:24,258 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:24,272 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:24,272 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:24,272 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:24,272 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:24,273 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:24,273 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:24,274 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:24,274 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DDB94BB5FAC64F289A166CD9E42CDE71, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:24,278 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796284026_0001_1_00_000000_0
2016-01-14 18:31:24,278 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DDB94BB5FAC64F289A166CD9E42CDE71, mem on close (mb), free: 180, total: 368, max: 672
2016-01-14 18:31:24,278 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:24,278 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:24,279 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:24,279 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:24,281 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:24 Completed running task attempt: attempt_1452796284026_0001_1_00_000000_0
2016-01-14 18:31:24,285 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796284026_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:24,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796284026_0001_1_00, eventCount=1
2016-01-14 18:31:24,286 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:24,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DDB94BB5FAC64F289A166CD9E42CDE71, taskAttemptId=attempt_1452796284026_0001_1_00_000000_0, startTime=1452796284230, finishTime=1452796284286, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796284274, Process_Duration=4, Process_End_Time=1452796284278, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:24,287 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284026_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:24,287 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:24,287 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796284026_0001_1_00_000000_0
2016-01-14 18:31:24,288 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:24,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:TASK_FINISHED]: vertexName=DDB94BB5FAC64F289A166CD9E42CDE71, taskId=task_1452796284026_0001_1_00_000000, startTime=1452796284230, finishTime=1452796284287, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1452796284026_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796284274, Process_Duration=4, Process_End_Time=1452796284278, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:24,288 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284026_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:24,288 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71] : 1
2016-01-14 18:31:24,288 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71]
2016-01-14 18:31:24,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:24,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:VERTEX_FINISHED]: vertexName=DDB94BB5FAC64F289A166CD9E42CDE71, vertexId=vertex_1452796284026_0001_1_00, initRequestedTime=1452796284194, initedTime=1452796284209, startRequestedTime=1452796284211, startedTime=1452796284211, finishTime=1452796284288, timeTaken=77, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796284274, Process_Duration=4, Process_End_Time=1452796284278, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796284230, firstTasksToStart=[ task_1452796284026_0001_1_00_000000 ], lastTaskFinishTime=1452796284286, lastTasksToFinish=[ task_1452796284026_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796284026_0001_1_00_000000 ], longestDurationTasks=[ task_1452796284026_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:24,290 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:24,290 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:24,290 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] attempt: attempt_1452796284026_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:24,290 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A78EE660D55044F5AA1DEE245FCE0705
2016-01-14 18:31:24,290 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705]
2016-01-14 18:31:24,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796284026_0001_00_000001, stoppedTime=1452796284291, exitStatus=0
2016-01-14 18:31:24,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:24,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796284274, Process_Duration=4, Process_End_Time=1452796284278, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:24,291 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:24,291 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:24,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:TASK_STARTED]: vertexName=A78EE660D55044F5AA1DEE245FCE0705, taskId=task_1452796284026_0001_1_01_000000, scheduledTime=1452796284292, launchTime=1452796284292
2016-01-14 18:31:24,292 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284026_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:24,292 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705]
2016-01-14 18:31:24,292 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796284026_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:24,292 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: A78EE660D55044F5AA1DEE245FCE0705, VertexParallelism: 1, TaskAttemptID:attempt_1452796284026_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DDB94BB5FAC64F289A166CD9E42CDE71, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=74D6DA6EA6E64EC69DE66A08681AEC62, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:24,292 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284026_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:24,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:24,293 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:24,293 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:24,292 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:24,293 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:24,293 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796284026_0001_00_000001
2016-01-14 18:31:24,293 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796284026_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:24,294 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796284026_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:24,294 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796284026_0001_00_000002
2016-01-14 18:31:24,295 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:24,295 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796284026_0001_00_000001, Exiting
2016-01-14 18:31:24,295 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796284026_0001_00_000001
2016-01-14 18:31:24,295 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796284026_0001_00_000001
2016-01-14 18:31:24,296 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796284026_0001_00_000002
2016-01-14 18:31:24,297 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:24,297 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796284026_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:24,298 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796284026_0001_00_000001 completed successfully
2016-01-14 18:31:24,298 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:24,299 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:24,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:24,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796284026_0001_00_000002, launchTime=1452796284299
2016-01-14 18:31:24,300 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796284026_0001_00_000002
2016-01-14 18:31:24,301 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796284026_0001_1_01_000000_0] to container: [container_1452796284026_0001_00_000002]
2016-01-14 18:31:24,301 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:24,301 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796284026_0001_00_000002 given task: attempt_1452796284026_0001_1_01_000000_0
2016-01-14 18:31:24,302 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796284026_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796284026_0001_1_01_000000_0
2016-01-14 18:31:24,302 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796284026_0001_1_01_000000_0] started. Is using containerId: [container_1452796284026_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:24,302 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:24,302 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A78EE660D55044F5AA1DEE245FCE0705, taskAttemptId=attempt_1452796284026_0001_1_01_000000_0, startTime=1452796284301, containerId=container_1452796284026_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796284026_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:24,302 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284026_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:24,302 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284026_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:24 Starting to run new task attempt: attempt_1452796284026_0001_1_01_000000_0
2016-01-14 18:31:24,304 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796284026_0001_1_01_000000_0
2016-01-14 18:31:24,304 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:24,304 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:24,304 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:24,304 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:24,305 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:24,305 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:24,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C] every/(1/1) ...latform/-1_VALUES_REPLACE, VertexName: A78EE660D55044F5AA1DEE245FCE0705, VertexParallelism: 1, TaskAttemptID:attempt_1452796284026_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DDB94BB5FAC64F289A166CD9E42CDE71, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=74D6DA6EA6E64EC69DE66A08681AEC62, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:24,305 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:24,306 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796284026_0001_1_01_000000_0
2016-01-14 18:31:24,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:24,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DDB94BB5FAC64F289A166CD9E42CDE71, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:24,307 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796284026_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:24,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:24,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DDB94BB5FAC64F289A166CD9E42CDE71
2016-01-14 18:31:24,310 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:24,310 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:24,311 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: DDB94BB5FAC64F289A166CD9E42CDE71
2016-01-14 18:31:24,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DDB94BB5FAC64F289A166CD9E42CDE71
2016-01-14 18:31:24,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,316 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=74D6DA6EA6E64EC69DE66A08681AEC62, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:24,316 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:24,316 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:24,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 74D6DA6EA6E64EC69DE66A08681AEC62
2016-01-14 18:31:24,317 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:24,318 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 74D6DA6EA6E64EC69DE66A08681AEC62
2016-01-14 18:31:24,325 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A78EE660D55044F5AA1DEE245FCE0705, ordinal: 1
2016-01-14 18:31:24,325 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A78EE660D55044F5AA1DEE245FCE0705, mem on start (mb), free: 173, total: 368, max: 672
2016-01-14 18:31:24,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:24,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:24,328 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:24,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 74D6DA6EA6E64EC69DE66A08681AEC62
2016-01-14 18:31:24,328 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:24,328 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:24,328 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:24,328 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:24,328 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:24,328 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:24,329 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:24,329 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DDB94BB5FAC64F289A166CD9E42CDE71, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:24,329 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 74D6DA6EA6E64EC69DE66A08681AEC62, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:24,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:24,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DDB94BB5FAC64F289A166CD9E42CDE71 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:24,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:24,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:24,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DDB94BB5FAC64F289A166CD9E42CDE71
2016-01-14 18:31:24,330 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:24,330 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:24,331 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: DDB94BB5FAC64F289A166CD9E42CDE71 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:24,331 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:24,331 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:24,332 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:24,333 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DDB94BB5FAC64F289A166CD9E42CDE71
2016-01-14 18:31:24,334 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:24,334 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796284026_0001_1_01_000000_0
2016-01-14 18:31:24,341 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796284026_0001_1_00_000000_0_10081, runDuration: 46154, ]
2016-01-14 18:31:24,342 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:24,343 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 964E8D10BA43412E8A3C4DFA3F11780A
2016-01-14 18:31:24,343 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE"], id: 74D6DA6EA6E64EC69DE66A08681AEC62
2016-01-14 18:31:24,343 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_REPLACE"]
2016-01-14 18:31:24,343 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 964E8D10BA43412E8A3C4DFA3F11780A, for 1 inputs
2016-01-14 18:31:24,345 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796284026_0001_1_00_000000_0_10081] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796284345, TimeTaken=2, Rate=0.14 MB/s
2016-01-14 18:31:24,345 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : DDB94BB5FAC64F289A166CD9E42CDE71
2016-01-14 18:31:24,346 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:24,346 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [DDB94BB5FAC64F289A166CD9E42CDE71] #14 in 4ms
2016-01-14 18:31:24,348 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [DDB94BB5FAC64F289A166CD9E42CDE71] #14, status:false, isInterrupted:false
2016-01-14 18:31:24,348 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:24,349 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:24,350 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:24,350 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:24,351 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:24,351 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : DDB94BB5FAC64F289A166CD9E42CDE71
2016-01-14 18:31:24,351 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A78EE660D55044F5AA1DEE245FCE0705, all 1 inputs ready in: 00:00:00.008
2016-01-14 18:31:24,352 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:24,353 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@523f560d; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3c2a99aa
2016-01-14 18:31:24,354 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:24,358 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:24,359 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:24,359 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:24,360 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796284026_0001_1_01_000000_0
2016-01-14 18:31:24,360 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796284026_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:24,361 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796284026_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:24,365 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A78EE660D55044F5AA1DEE245FCE0705, taskAttemptId=attempt_1452796284026_0001_1_01_000000_0, startTime=1452796284301, finishTime=1452796284365, timeTaken=64, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=21, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796284352, Process_Duration=9, Process_End_Time=1452796284361, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:24,365 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284026_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:24,366 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796284026_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:24,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:TASK_FINISHED]: vertexName=A78EE660D55044F5AA1DEE245FCE0705, taskId=task_1452796284026_0001_1_01_000000, startTime=1452796284301, finishTime=1452796284366, timeTaken=65, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:24,366 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:24,367 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284026_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:24,367 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] : 1
2016-01-14 18:31:24,367 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] because task failed: task_1452796284026_0001_1_01_000000
2016-01-14 18:31:24,368 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:24,368 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:24,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:24,368 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:24,368 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705]
2016-01-14 18:31:24,368 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=74D6DA6EA6E64EC69DE66A08681AEC62, vertexId=vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705]
2016-01-14 18:31:24,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:24,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:VERTEX_FINISHED]: vertexName=A78EE660D55044F5AA1DEE245FCE0705, vertexId=vertex_1452796284026_0001_1_01, initRequestedTime=1452796284196, initedTime=1452796284205, startRequestedTime=1452796284217, startedTime=1452796284217, finishTime=1452796284369, timeTaken=152, status=FAILED, diagnostics=Task failed, taskId=task_1452796284026_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796284301, firstTasksToStart=[ task_1452796284026_0001_1_01_000000 ], lastTaskFinishTime=1452796284365, lastTasksToFinish=[ task_1452796284026_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:24,370 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:24,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796284026_0001_00_000002, stoppedTime=1452796284374, exitStatus=0
2016-01-14 18:31:24,375 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:24,375 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796284026_0001_1_01_000000_0 failed: true container: container_1452796284026_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:24,375 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:24,375 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:24,375 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796284026_0001_1
2016-01-14 18:31:24,375 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796284026_0001_1_00 [DDB94BB5FAC64F289A166CD9E42CDE71]
2016-01-14 18:31:24,375 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:24,376 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 74D6DA6EA6E64EC69DE66A08681AEC62 for vertex: vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705]
2016-01-14 18:31:24,378 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796284026_0001_1, queueSize=0
2016-01-14 18:31:24,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:24,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=16, MERGE_PHASE_TIME=21, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796284352, Process_Duration=9, Process_End_Time=1452796284361, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:24,381 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A78EE660D55044F5AA1DEE245FCE0705, mem on close (mb), free: 168, total: 368, max: 672
2016-01-14 18:31:24,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=A78EE660D55044F5AA1DEE245FCE0705, index=1
2016-01-14 18:31:24,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284026_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796284026_0001_1, startTime=1452796284194, finishTime=1452796284376, timeTaken=182, status=FAILED, diagnostics=Vertex failed, vertexName=A78EE660D55044F5AA1DEE245FCE0705, vertexId=vertex_1452796284026_0001_1_01, diagnostics=[Task failed, taskId=task_1452796284026_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796284026_0001_1_01 [A78EE660D55044F5AA1DEE245FCE0705] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796284274, Process_Duration=4, Process_End_Time=1452796284278, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:24,382 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796284026_0001_1 finished with state: FAILED
2016-01-14 18:31:24,384 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796284026_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:24,384 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: DDB94BB5FAC64F289A166CD9E42CDE71
2016-01-14 18:31:24 Completed Dag: dag_1452796284026_0001_1
2016-01-14 18:31:24,385 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796284026_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:24,385 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796284026_0001_1, dagState=FAILED
2016-01-14 18:31:24,387 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796284026_0001_1_post
2016-01-14 18:31:24,387 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:24,387 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:24,387 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:24,385 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:24,388 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:24,388 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:24,388 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:24,388 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:24,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=A78EE660D55044F5AA1DEE245FCE0705, sourceVertex=DDB94BB5FAC64F289A166CD9E42CDE71
2016-01-14 18:31:24,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=A78EE660D55044F5AA1DEE245FCE0705, sourceVertex=74D6DA6EA6E64EC69DE66A08681AEC62
2016-01-14 18:31:24,390 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796284026_0001_00_000002
2016-01-14 18:31:24,390 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796284026_0001_00_000002
2016-01-14 18:31:24,391 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796284026_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:24,391 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796284026_0001_00_000002 completed but with errors
2016-01-14 18:31:24,391 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796284026_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:24,391 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284026_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:24,391 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284026_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:24,392 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/-1_VALUES_REPLACE, applicationId=application_1452796284026_0001
2016-01-14 18:31:24,394 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:24,394 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:24,395 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:24,395 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:24,395 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:24,396 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:24,396 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:24,397 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:24,397 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:24,397 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:24,397 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:24,397 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:24,397 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:24,403 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/2A8CBD8F7D8E4B478EEC4E32F1D3FB2D/EF943BC952804189B9B585637234EA8C/.tez/application_1452796284026_0001
2016-01-14 18:31:24,403 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:24,404 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...latform/-1_VALUES_REPLACE
2016-01-14 18:31:24,404 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:24,405 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.400
2016-01-14 18:31:24,405 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_REPLACE
2016-01-14 18:31:24,436 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:24,436 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:24,476 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 18:31:24,478 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 18:31:24,479 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:24,479 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:24,479 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:24,495 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:24,501 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP
2016-01-14 18:31:24,559 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:24,559 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:24,559 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP"]
2016-01-14 18:31:24,560 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:24,560 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:24,560 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:24,561 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 18:31:24,562 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:24,576 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:24,581 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:24,581 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:24,581 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:24,590 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653/.tez/application_1452796284581_0001 doesn't exist and is created
2016-01-14 18:31:24,605 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:24,605 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:24,606 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653/.tez/application_1452796284581_0001
2016-01-14 18:31:24,608 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796284581_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:24,619 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:24,620 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:24,622 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:24,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:24,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653/.tez/application_1452796284581_0001/localmode-log-dir
2016-01-14 18:31:24,623 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653/.tez/application_1452796284581_0001/localmode-log-dir/history.txt.appattempt_1452796284581_0001_000000, maxErrors=10
2016-01-14 18:31:24,623 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:24,624 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796284581_0001_000000, appSubmitTime=1452796284607, launchTime=1452796284608
2016-01-14 18:31:24,627 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:24,628 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:24,629 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:24,629 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:24,633 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:40948
2016-01-14 18:31:24,634 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:24,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796284581_0001_000000, startTime=1452796284636
2016-01-14 18:31:24,636 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:24,636 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:24,705 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:24,705 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:24,705 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1452796284581_0001, dagName=[E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 18:31:24,722 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 18:31:24,723 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796284581_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653/.tez/application_1452796284581_0001/localmode-log-dir/dag_1452796284581_0001_1.dot
2016-01-14 18:31:24,724 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653/.tez/application_1452796284581_0001/localmode-log-dir/dag_1452796284581_0001_1-tez-dag.pb.txt
2016-01-14 18:31:24,737 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796284581_0001_1
2016-01-14 18:31:24,737 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:24,738 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 18:31:24 Running Dag: dag_1452796284581_0001_1
2016-01-14 18:31:24,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796284581_0001_1, submitTime=1452796284722
2016-01-14 18:31:24,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:24,750 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DD76E77964774C028E5666CD1D13A15C
2016-01-14 18:31:24,751 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:24,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796284581_0001_1, initTime=1452796284750
2016-01-14 18:31:24,751 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796284581_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:24,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:24,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:DAG_STARTED]: dagID=dag_1452796284581_0001_1, startTime=1452796284751
2016-01-14 18:31:24,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:24,751 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:24,752 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796284581_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:24,752 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FB2F0D66F95647C795D0242BA95606AA : {72971829358B488BA59BC37787BF58DE={InputName=72971829358B488BA59BC37787BF58DE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:24,753 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 72971829358B488BA59BC37787BF58DE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:24,753 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA]
2016-01-14 18:31:24,753 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA]
2016-01-14 18:31:24,753 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA]
2016-01-14 18:31:24,753 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1452796284581_0001, dagName=[E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653] every/(1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 18:31:24,753 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA]. Starting root input initializers: 1
2016-01-14 18:31:24,754 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:24,754 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C]
2016-01-14 18:31:24,755 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 72971829358B488BA59BC37787BF58DE on vertex vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA]
2016-01-14 18:31:24,757 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:24,757 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C]
2016-01-14 18:31:24,757 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C]
2016-01-14 18:31:24,758 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C]
2016-01-14 18:31:24,758 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C2D4CD1A404E45DA9C6EC0B37DF4A0E0, vertexId=vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:24,758 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C2D4CD1A404E45DA9C6EC0B37DF4A0E0, vertexId=vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C]
2016-01-14 18:31:24,760 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:24,760 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C2D4CD1A404E45DA9C6EC0B37DF4A0E0, vertexName=DD76E77964774C028E5666CD1D13A15C, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:24,760 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C2D4CD1A404E45DA9C6EC0B37DF4A0E0, vertexId=vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C]
2016-01-14 18:31:24,760 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 72971829358B488BA59BC37787BF58DE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:24,761 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:24,761 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:24,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD76E77964774C028E5666CD1D13A15C, vertexId=vertex_1452796284581_0001_1_01, initRequestedTime=1452796284754, initedTime=1452796284761, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:24,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:24,762 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:24,766 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:24,767 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 72971829358B488BA59BC37787BF58DE on vertex vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA]
2016-01-14 18:31:24,767 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:24,767 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {72971829358B488BA59BC37787BF58DE=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:24,768 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA] parallelism set to 1
2016-01-14 18:31:24,768 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:24,768 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796284581_0001_1_00, eventCount=1
2016-01-14 18:31:24,769 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:24,769 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FB2F0D66F95647C795D0242BA95606AA, vertexId=vertex_1452796284581_0001_1_00, initRequestedTime=1452796284752, initedTime=1452796284768, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:24,769 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA] with distanceFromRoot: 0
2016-01-14 18:31:24,770 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:24,770 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FB2F0D66F95647C795D0242BA95606AA
2016-01-14 18:31:24,770 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA]
2016-01-14 18:31:24,771 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:24,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796284581_0001_1_00, startRequestedTime=1452796284770, startedTime=1452796284770
2016-01-14 18:31:24,771 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:24,772 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:24,772 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:TASK_STARTED]: vertexName=FB2F0D66F95647C795D0242BA95606AA, taskId=task_1452796284581_0001_1_00_000000, scheduledTime=1452796284772, launchTime=1452796284772
2016-01-14 18:31:24,772 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284581_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:24,773 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796284581_0001_1_00 for vertex: vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] numStartedSources: 1 numSources: 1
2016-01-14 18:31:24,773 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] with distanceFromRoot: 1
2016-01-14 18:31:24,773 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA]
2016-01-14 18:31:24,773 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FB2F0D66F95647C795D0242BA95606AA in vertex: DD76E77964774C028E5666CD1D13A15C
2016-01-14 18:31:24,774 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DD76E77964774C028E5666CD1D13A15C with 1 source tasks and 1 pending tasks
2016-01-14 18:31:24,774 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:24,774 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796284581_0001_1_01, startRequestedTime=1452796284773, startedTime=1452796284773
2016-01-14 18:31:24,775 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:24,775 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: FB2F0D66F95647C795D0242BA95606AA, VertexParallelism: 1, TaskAttemptID:attempt_1452796284581_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=72971829358B488BA59BC37787BF58DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD76E77964774C028E5666CD1D13A15C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:24,775 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284581_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:24,775 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:24,776 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:24,776 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:24,777 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:24,777 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:24,777 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:24,777 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796284581_0001_00_000001
2016-01-14 18:31:24,778 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:24,780 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796284581_0001_00_000001
2016-01-14 18:31:24,780 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:24,786 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796284581_0001, with dag id: dag_1452796284581_0001_1
2016-01-14 18:31:24,787 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:24,788 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:24,788 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796284581_0001_00_000001, launchTime=1452796284787
2016-01-14 18:31:24,789 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796284581_0001_00_000001
2016-01-14 18:31:24,790 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796284581_0001_1_00_000000_0] to container: [container_1452796284581_0001_00_000001]
2016-01-14 18:31:24,790 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:24,791 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796284581_0001_1_00_000000_0] started. Is using containerId: [container_1452796284581_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:24,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FB2F0D66F95647C795D0242BA95606AA, taskAttemptId=attempt_1452796284581_0001_1_00_000000_0, startTime=1452796284791, containerId=container_1452796284581_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796284581_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:24,791 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284581_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:24,791 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284581_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:24,792 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:24,793 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796284581_0001_00_000001 given task: attempt_1452796284581_0001_1_00_000000_0
2016-01-14 18:31:24,793 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796284581_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796284581_0001_1_00_000000_0
2016-01-14 18:31:24 Starting to run new task attempt: attempt_1452796284581_0001_1_00_000000_0
2016-01-14 18:31:24,794 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796284581_0001_1_00_000000_0
2016-01-14 18:31:24,794 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:24,794 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:24,794 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:24,795 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:24,795 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:24,795 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:24,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: FB2F0D66F95647C795D0242BA95606AA, VertexParallelism: 1, TaskAttemptID:attempt_1452796284581_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=72971829358B488BA59BC37787BF58DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD76E77964774C028E5666CD1D13A15C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:24,795 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:24,797 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796284581_0001_1_00_000000_0
2016-01-14 18:31:24,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:24,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=72971829358B488BA59BC37787BF58DE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:24,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DD76E77964774C028E5666CD1D13A15C, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:24,804 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796284581_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:24,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,807 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:24,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:24,809 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DD76E77964774C028E5666CD1D13A15C
2016-01-14 18:31:24,811 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:24,811 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DD76E77964774C028E5666CD1D13A15C
2016-01-14 18:31:24,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DD76E77964774C028E5666CD1D13A15C
2016-01-14 18:31:24,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:24,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 72971829358B488BA59BC37787BF58DE
2016-01-14 18:31:24,813 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 72971829358B488BA59BC37787BF58DE
2016-01-14 18:31:24,814 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FB2F0D66F95647C795D0242BA95606AA, ordinal: 0
2016-01-14 18:31:24,814 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FB2F0D66F95647C795D0242BA95606AA, mem on start (mb), free: 288, total: 369, max: 672
2016-01-14 18:31:24,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:24,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:24,817 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:24,817 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 72971829358B488BA59BC37787BF58DE
2016-01-14 18:31:24,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 72971829358B488BA59BC37787BF58DE
2016-01-14 18:31:24,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:24,818 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:24,818 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:24,818 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:24,818 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:24,819 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:24,819 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:24,819 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DD76E77964774C028E5666CD1D13A15C, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:24,819 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 72971829358B488BA59BC37787BF58DE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:24,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:24,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 72971829358B488BA59BC37787BF58DE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:24,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:24,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:24,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 72971829358B488BA59BC37787BF58DE
2016-01-14 18:31:24,821 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:24,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 72971829358B488BA59BC37787BF58DE
2016-01-14 18:31:24,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:24,822 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796284581_0001_1_00_000000_0
2016-01-14 18:31:24,822 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:24,822 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5adbc61e
2016-01-14 18:31:24,824 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5adbc61e
2016-01-14 18:31:24,824 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:24,824 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:24,826 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:24,828 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 72971829358B488BA59BC37787BF58DE
2016-01-14 18:31:24,830 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: B733030720F24771916DC79B1E7B3F14
2016-01-14 18:31:24,830 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] B733030720F24771916DC79B1E7B3F14
2016-01-14 18:31:24,831 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@f2151d9; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@54848551; comparator=cascading.tuple.hadoop.util.TupleComparator@484cb430; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:24,831 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:24,832 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:24,832 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:24,847 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:24,848 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:24,848 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:24,848 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:24,848 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:24,850 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:24,850 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:24,850 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FB2F0D66F95647C795D0242BA95606AA, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:24,853 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796284581_0001_1_00_000000_0
2016-01-14 18:31:24,853 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FB2F0D66F95647C795D0242BA95606AA, mem on close (mb), free: 184, total: 369, max: 672
2016-01-14 18:31:24,853 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:24,853 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:24,854 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:24,854 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:24,856 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:24 Completed running task attempt: attempt_1452796284581_0001_1_00_000000_0
2016-01-14 18:31:24,860 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796284581_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:24,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796284581_0001_1_00, eventCount=1
2016-01-14 18:31:24,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FB2F0D66F95647C795D0242BA95606AA, taskAttemptId=attempt_1452796284581_0001_1_00_000000_0, startTime=1452796284791, finishTime=1452796284861, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796284850, Process_Duration=3, Process_End_Time=1452796284853, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:24,861 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284581_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:24,862 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:24,862 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796284581_0001_1_00_000000_0
2016-01-14 18:31:24,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:TASK_FINISHED]: vertexName=FB2F0D66F95647C795D0242BA95606AA, taskId=task_1452796284581_0001_1_00_000000, startTime=1452796284791, finishTime=1452796284862, timeTaken=71, status=SUCCEEDED, successfulAttemptID=attempt_1452796284581_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796284850, Process_Duration=3, Process_End_Time=1452796284853, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:24,863 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284581_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:24,863 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA] : 1
2016-01-14 18:31:24,863 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA]
2016-01-14 18:31:24,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:24,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:VERTEX_FINISHED]: vertexName=FB2F0D66F95647C795D0242BA95606AA, vertexId=vertex_1452796284581_0001_1_00, initRequestedTime=1452796284752, initedTime=1452796284768, startRequestedTime=1452796284770, startedTime=1452796284770, finishTime=1452796284863, timeTaken=93, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796284850, Process_Duration=3, Process_End_Time=1452796284853, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796284791, firstTasksToStart=[ task_1452796284581_0001_1_00_000000 ], lastTaskFinishTime=1452796284861, lastTasksToFinish=[ task_1452796284581_0001_1_00_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796284581_0001_1_00_000000 ], longestDurationTasks=[ task_1452796284581_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:24,864 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:24,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:24,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:24,866 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:24,866 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] attempt: attempt_1452796284581_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:24,867 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DD76E77964774C028E5666CD1D13A15C
2016-01-14 18:31:24,867 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C]
2016-01-14 18:31:24,867 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:24,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796284581_0001_00_000001, stoppedTime=1452796284867, exitStatus=0
2016-01-14 18:31:24,867 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:24,868 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:TASK_STARTED]: vertexName=DD76E77964774C028E5666CD1D13A15C, taskId=task_1452796284581_0001_1_01_000000, scheduledTime=1452796284868, launchTime=1452796284868
2016-01-14 18:31:24,868 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284581_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:24,868 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C]
2016-01-14 18:31:24,868 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: DD76E77964774C028E5666CD1D13A15C, VertexParallelism: 1, TaskAttemptID:attempt_1452796284581_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FB2F0D66F95647C795D0242BA95606AA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C2D4CD1A404E45DA9C6EC0B37DF4A0E0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:24,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:24,868 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796284581_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:24,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:24,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796284850, Process_Duration=3, Process_End_Time=1452796284853, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:24,869 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284581_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:24,869 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:24,869 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:24,870 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:24,870 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796284581_0001_00_000001
2016-01-14 18:31:24,870 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796284581_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:24,871 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796284581_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:24,870 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:24,872 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796284581_0001_00_000002
2016-01-14 18:31:24,872 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796284581_0001_00_000001, Exiting
2016-01-14 18:31:24,873 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796284581_0001_00_000001
2016-01-14 18:31:24,873 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796284581_0001_00_000001
2016-01-14 18:31:24,874 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796284581_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:24,874 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796284581_0001_00_000001 completed successfully
2016-01-14 18:31:24,875 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:24,876 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:24,877 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796284581_0001_00_000002
2016-01-14 18:31:24,878 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:24,880 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:24,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:24,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796284581_0001_00_000002, launchTime=1452796284880
2016-01-14 18:31:24,882 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796284581_0001_00_000002
2016-01-14 18:31:24,882 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796284581_0001_1_01_000000_0] to container: [container_1452796284581_0001_00_000002]
2016-01-14 18:31:24,883 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:24,883 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796284581_0001_00_000002 given task: attempt_1452796284581_0001_1_01_000000_0
2016-01-14 18:31:24,883 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796284581_0001_1_01_000000_0] started. Is using containerId: [container_1452796284581_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:24,884 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796284581_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796284581_0001_1_01_000000_0
2016-01-14 18:31:24,884 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD76E77964774C028E5666CD1D13A15C, taskAttemptId=attempt_1452796284581_0001_1_01_000000_0, startTime=1452796284883, containerId=container_1452796284581_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796284581_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:24,884 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:24,884 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284581_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:24 Starting to run new task attempt: attempt_1452796284581_0001_1_01_000000_0
2016-01-14 18:31:24,885 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284581_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:24,885 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796284581_0001_1_01_000000_0
2016-01-14 18:31:24,885 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:24,885 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:24,886 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:24,886 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:24,886 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:24,886 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:24,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653] every/(1/1) ...lyplatform/-1_VALUES_SWAP, VertexName: DD76E77964774C028E5666CD1D13A15C, VertexParallelism: 1, TaskAttemptID:attempt_1452796284581_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FB2F0D66F95647C795D0242BA95606AA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C2D4CD1A404E45DA9C6EC0B37DF4A0E0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:24,887 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:24,887 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796284581_0001_1_01_000000_0
2016-01-14 18:31:24,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:24,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FB2F0D66F95647C795D0242BA95606AA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:24,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C2D4CD1A404E45DA9C6EC0B37DF4A0E0, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:24,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:24,893 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C2D4CD1A404E45DA9C6EC0B37DF4A0E0
2016-01-14 18:31:24,893 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:24,893 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C2D4CD1A404E45DA9C6EC0B37DF4A0E0
2016-01-14 18:31:24,892 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796284581_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:24,892 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:24,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FB2F0D66F95647C795D0242BA95606AA
2016-01-14 18:31:24,897 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:24,899 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:24,899 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:24,899 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FB2F0D66F95647C795D0242BA95606AA
2016-01-14 18:31:24,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FB2F0D66F95647C795D0242BA95606AA
2016-01-14 18:31:24,904 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:24,904 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C2D4CD1A404E45DA9C6EC0B37DF4A0E0
2016-01-14 18:31:24,906 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD76E77964774C028E5666CD1D13A15C, ordinal: 1
2016-01-14 18:31:24,906 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD76E77964774C028E5666CD1D13A15C, mem on start (mb), free: 177, total: 369, max: 672
2016-01-14 18:31:24,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:24,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:24,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:24,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:24,906 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:24,907 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:24,907 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:24,907 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:24,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:24,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:24,908 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C2D4CD1A404E45DA9C6EC0B37DF4A0E0, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:24,908 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FB2F0D66F95647C795D0242BA95606AA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:24,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:24,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FB2F0D66F95647C795D0242BA95606AA
2016-01-14 18:31:24,909 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FB2F0D66F95647C795D0242BA95606AA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:24,909 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:24,910 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:24,910 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FB2F0D66F95647C795D0242BA95606AA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:24,910 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:24,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:24,911 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:24,911 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:24,911 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:24,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FB2F0D66F95647C795D0242BA95606AA
2016-01-14 18:31:24,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:24,913 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796284581_0001_1_01_000000_0
2016-01-14 18:31:24,914 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796284581_0001_1_00_000000_0_10087, runDuration: 49186, ]
2016-01-14 18:31:24,915 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796284581_0001_1_00_000000_0_10087] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796284915, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:24,915 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FB2F0D66F95647C795D0242BA95606AA
2016-01-14 18:31:24,915 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:24,916 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FB2F0D66F95647C795D0242BA95606AA] #15 in 2ms
2016-01-14 18:31:24,916 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FB2F0D66F95647C795D0242BA95606AA] #15, status:false, isInterrupted:false
2016-01-14 18:31:24,917 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:24,919 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:24,920 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:24,920 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:24,920 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: B733030720F24771916DC79B1E7B3F14
2016-01-14 18:31:24,921 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP"], id: C2D4CD1A404E45DA9C6EC0B37DF4A0E0
2016-01-14 18:31:24,921 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_VALUES_SWAP"]
2016-01-14 18:31:24,922 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] B733030720F24771916DC79B1E7B3F14, for 1 inputs
2016-01-14 18:31:24,921 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:24,924 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:24,924 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FB2F0D66F95647C795D0242BA95606AA
2016-01-14 18:31:24,925 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:24,924 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD76E77964774C028E5666CD1D13A15C, all 1 inputs ready in: 00:00:00.001
2016-01-14 18:31:24,925 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6907e6cb; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5c14f4c
2016-01-14 18:31:24,926 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:24,927 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:24,927 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:24,928 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:24,928 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796284581_0001_1_01_000000_0
2016-01-14 18:31:24,928 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796284581_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:24,929 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796284581_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:24,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:24,934 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796284925, Process_Duration=4, Process_End_Time=1452796284929, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:24,938 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD76E77964774C028E5666CD1D13A15C, mem on close (mb), free: 170, total: 369, max: 672
2016-01-14 18:31:24,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=DD76E77964774C028E5666CD1D13A15C, index=1
2016-01-14 18:31:24,934 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD76E77964774C028E5666CD1D13A15C, taskAttemptId=attempt_1452796284581_0001_1_01_000000_0, startTime=1452796284883, finishTime=1452796284933, timeTaken=50, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796284925, Process_Duration=4, Process_End_Time=1452796284929, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:24,939 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284581_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:24,939 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796284581_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:24,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:TASK_FINISHED]: vertexName=DD76E77964774C028E5666CD1D13A15C, taskId=task_1452796284581_0001_1_01_000000, startTime=1452796284883, finishTime=1452796284939, timeTaken=56, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:24,940 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:24,940 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796284581_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:24,940 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] : 1
2016-01-14 18:31:24,943 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FB2F0D66F95647C795D0242BA95606AA
2016-01-14 18:31:24,943 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:24,943 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] because task failed: task_1452796284581_0001_1_01_000000
2016-01-14 18:31:24,943 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:24,943 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:24,943 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:24,944 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:24,944 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:24,944 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C]
2016-01-14 18:31:24,944 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=C2D4CD1A404E45DA9C6EC0B37DF4A0E0, vertexId=vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C]
2016-01-14 18:31:24,944 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:24,944 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:24,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=DD76E77964774C028E5666CD1D13A15C, sourceVertex=FB2F0D66F95647C795D0242BA95606AA
2016-01-14 18:31:24,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=DD76E77964774C028E5666CD1D13A15C, sourceVertex=C2D4CD1A404E45DA9C6EC0B37DF4A0E0
2016-01-14 18:31:24,945 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796284581_0001_00_000002
2016-01-14 18:31:24,945 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796284581_0001_00_000002
2016-01-14 18:31:24,946 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:24,946 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796284581_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:24,946 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796284581_0001_00_000002 completed but with errors
2016-01-14 18:31:24,946 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:VERTEX_FINISHED]: vertexName=DD76E77964774C028E5666CD1D13A15C, vertexId=vertex_1452796284581_0001_1_01, initRequestedTime=1452796284754, initedTime=1452796284761, startRequestedTime=1452796284773, startedTime=1452796284773, finishTime=1452796284945, timeTaken=172, status=FAILED, diagnostics=Task failed, taskId=task_1452796284581_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796284883, firstTasksToStart=[ task_1452796284581_0001_1_01_000000 ], lastTaskFinishTime=1452796284933, lastTasksToFinish=[ task_1452796284581_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:24,947 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:24,947 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:24,947 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796284581_0001_00_000002, stoppedTime=1452796284947, exitStatus=0
2016-01-14 18:31:24,947 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:24,947 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796284581_0001_1_01_000000_0 failed: true container: container_1452796284581_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:24,947 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:24,948 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:24,948 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796284581_0001_1
2016-01-14 18:31:24,948 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796284581_0001_1_00 [FB2F0D66F95647C795D0242BA95606AA]
2016-01-14 18:31:24,948 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: C2D4CD1A404E45DA9C6EC0B37DF4A0E0 for vertex: vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C]
2016-01-14 18:31:24,950 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796284581_0001_1, queueSize=0
2016-01-14 18:31:24,950 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:24,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796284581_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796284581_0001_1, startTime=1452796284751, finishTime=1452796284948, timeTaken=197, status=FAILED, diagnostics=Vertex failed, vertexName=DD76E77964774C028E5666CD1D13A15C, vertexId=vertex_1452796284581_0001_1_01, diagnostics=[Task failed, taskId=task_1452796284581_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796284581_0001_1_01 [DD76E77964774C028E5666CD1D13A15C] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796284850, Process_Duration=3, Process_End_Time=1452796284853, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:24,951 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796284581_0001_1 finished with state: FAILED
2016-01-14 18:31:24,951 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796284581_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:24,951 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796284581_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:24,952 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796284581_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:24,952 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796284581_0001_00_000002
2016-01-14 18:31:24 Completed Dag: dag_1452796284581_0001_1
2016-01-14 18:31:24,952 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796284581_0001_1, dagState=FAILED
2016-01-14 18:31:24,952 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796284581_0001_1_post
2016-01-14 18:31:24,952 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:24,952 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:24,953 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796284581_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:24,957 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/-1_VALUES_SWAP, applicationId=application_1452796284581_0001
2016-01-14 18:31:24,958 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:24,959 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:24,959 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:24,959 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:24,959 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:24,959 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:24,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:24,960 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:24,960 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:24,960 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:24,961 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:24,961 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:24,961 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:24,967 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E78EADA3BC064513AD08D53788E77F99/D61218B0AFB14A379855F88BAB7BA653/.tez/application_1452796284581_0001
2016-01-14 18:31:24,967 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:24,968 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...lyplatform/-1_VALUES_SWAP
2016-01-14 18:31:24,968 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:24,969 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.409
2016-01-14 18:31:24,969 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_VALUES_SWAP
2016-01-14 18:31:24,992 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:24,992 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:25,028 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 18:31:25,038 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-01-14 18:31:25,038 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:25,038 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:25,039 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:25,053 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:25,059 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1
2016-01-14 18:31:25,110 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:25,111 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:25,111 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1"]
2016-01-14 18:31:25,111 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:25,111 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:25,112 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:25,113 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...semblyplatform/-1_ARGS_-1
2016-01-14 18:31:25,113 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:25,138 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:25,143 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:25,144 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:25,144 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:25,152 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67/.tez/application_1452796285143_0001 doesn't exist and is created
2016-01-14 18:31:25,169 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:25,169 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:25,170 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67/.tez/application_1452796285143_0001
2016-01-14 18:31:25,171 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796285143_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:25,183 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:25,184 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:25,185 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:25,185 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:25,186 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67/.tez/application_1452796285143_0001/localmode-log-dir
2016-01-14 18:31:25,186 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67/.tez/application_1452796285143_0001/localmode-log-dir/history.txt.appattempt_1452796285143_0001_000000, maxErrors=10
2016-01-14 18:31:25,186 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:25,187 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796285143_0001_000000, appSubmitTime=1452796285171, launchTime=1452796285171
2016-01-14 18:31:25,193 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:25,193 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:25,194 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:25,194 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:25,196 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:38282
2016-01-14 18:31:25,199 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:25,202 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796285143_0001_000000, startTime=1452796285202
2016-01-14 18:31:25,202 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:25,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:25,270 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:25,270 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:25,270 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1452796285143_0001, dagName=[D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-01-14 18:31:25,297 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-01-14 18:31:25,299 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796285143_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67/.tez/application_1452796285143_0001/localmode-log-dir/dag_1452796285143_0001_1.dot
2016-01-14 18:31:25,300 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67/.tez/application_1452796285143_0001/localmode-log-dir/dag_1452796285143_0001_1-tez-dag.pb.txt
2016-01-14 18:31:25,321 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796285143_0001_1
2016-01-14 18:31:25,321 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:25,321 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-01-14 18:31:25 Running Dag: dag_1452796285143_0001_1
2016-01-14 18:31:25,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:25,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796285143_0001_1, submitTime=1452796285297
2016-01-14 18:31:25,333 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D8C3760D843D4C79B4044B9739F29EF4
2016-01-14 18:31:25,334 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:25,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:25,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796285143_0001_1, initTime=1452796285332
2016-01-14 18:31:25,336 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796285143_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:25,337 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:DAG_STARTED]: dagID=dag_1452796285143_0001_1, startTime=1452796285336
2016-01-14 18:31:25,337 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:25,337 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796285143_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:25,337 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EDAF121338594169AB05786F9781F36A : {30BCA951C61841E28202694F92D62A02={InputName=30BCA951C61841E28202694F92D62A02}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:25,337 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 30BCA951C61841E28202694F92D62A02, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:25,337 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A]
2016-01-14 18:31:25,337 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A]
2016-01-14 18:31:25,337 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A]
2016-01-14 18:31:25,338 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A]. Starting root input initializers: 1
2016-01-14 18:31:25,338 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:25,338 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:25,338 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4]
2016-01-14 18:31:25,343 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:25,343 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4]
2016-01-14 18:31:25,343 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4]
2016-01-14 18:31:25,343 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4]
2016-01-14 18:31:25,343 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7EB2DCB408194CE9B6A4948DA3D6E073, vertexId=vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:25,344 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7EB2DCB408194CE9B6A4948DA3D6E073, vertexId=vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4]
2016-01-14 18:31:25,346 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 30BCA951C61841E28202694F92D62A02 on vertex vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A]
2016-01-14 18:31:25,346 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1452796285143_0001, dagName=[D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67] every/(1/1) ...semblyplatform/-1_ARGS_-1
2016-01-14 18:31:25,347 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7EB2DCB408194CE9B6A4948DA3D6E073, vertexName=D8C3760D843D4C79B4044B9739F29EF4, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:25,348 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7EB2DCB408194CE9B6A4948DA3D6E073, vertexId=vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4]
2016-01-14 18:31:25,349 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:25,351 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:25,351 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 30BCA951C61841E28202694F92D62A02 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:25,351 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:25,352 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:25,353 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:25,354 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 30BCA951C61841E28202694F92D62A02 on vertex vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A]
2016-01-14 18:31:25,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D8C3760D843D4C79B4044B9739F29EF4, vertexId=vertex_1452796285143_0001_1_01, initRequestedTime=1452796285338, initedTime=1452796285349, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:25,354 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:25,355 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:25,355 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {30BCA951C61841E28202694F92D62A02=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:25,355 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A] parallelism set to 1
2016-01-14 18:31:25,356 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:25,356 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796285143_0001_1_00, eventCount=1
2016-01-14 18:31:25,357 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EDAF121338594169AB05786F9781F36A, vertexId=vertex_1452796285143_0001_1_00, initRequestedTime=1452796285337, initedTime=1452796285356, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:25,357 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A] with distanceFromRoot: 0
2016-01-14 18:31:25,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:25,357 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:25,358 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EDAF121338594169AB05786F9781F36A
2016-01-14 18:31:25,358 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A]
2016-01-14 18:31:25,358 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:25,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796285143_0001_1_00, startRequestedTime=1452796285358, startedTime=1452796285358
2016-01-14 18:31:25,359 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:25,359 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:25,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:TASK_STARTED]: vertexName=EDAF121338594169AB05786F9781F36A, taskId=task_1452796285143_0001_1_00_000000, scheduledTime=1452796285359, launchTime=1452796285359
2016-01-14 18:31:25,360 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285143_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:25,360 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796285143_0001_1_00 for vertex: vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4] numStartedSources: 1 numSources: 1
2016-01-14 18:31:25,360 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4] with distanceFromRoot: 1
2016-01-14 18:31:25,361 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A]
2016-01-14 18:31:25,361 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EDAF121338594169AB05786F9781F36A in vertex: D8C3760D843D4C79B4044B9739F29EF4
2016-01-14 18:31:25,361 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D8C3760D843D4C79B4044B9739F29EF4 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:25,362 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:25,362 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796285143_0001_1_01, startRequestedTime=1452796285361, startedTime=1452796285361
2016-01-14 18:31:25,362 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:25,363 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: EDAF121338594169AB05786F9781F36A, VertexParallelism: 1, TaskAttemptID:attempt_1452796285143_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=30BCA951C61841E28202694F92D62A02, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D8C3760D843D4C79B4044B9739F29EF4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:25,363 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285143_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:25,363 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:25,364 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:25,364 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:25,365 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:25,365 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796285143_0001_00_000001
2016-01-14 18:31:25,366 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:25,367 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796285143_0001_00_000001
2016-01-14 18:31:25,368 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:25,369 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796285143_0001, with dag id: dag_1452796285143_0001_1
2016-01-14 18:31:25,379 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:25,379 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:25,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796285143_0001_00_000001, launchTime=1452796285379
2016-01-14 18:31:25,381 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796285143_0001_00_000001
2016-01-14 18:31:25,381 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796285143_0001_1_00_000000_0] to container: [container_1452796285143_0001_00_000001]
2016-01-14 18:31:25,381 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:25,382 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796285143_0001_1_00_000000_0] started. Is using containerId: [container_1452796285143_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:25,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EDAF121338594169AB05786F9781F36A, taskAttemptId=attempt_1452796285143_0001_1_00_000000_0, startTime=1452796285382, containerId=container_1452796285143_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796285143_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:25,382 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285143_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:25,382 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285143_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:25,382 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:25,383 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796285143_0001_00_000001 given task: attempt_1452796285143_0001_1_00_000000_0
2016-01-14 18:31:25,383 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796285143_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796285143_0001_1_00_000000_0
2016-01-14 18:31:25 Starting to run new task attempt: attempt_1452796285143_0001_1_00_000000_0
2016-01-14 18:31:25,384 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796285143_0001_1_00_000000_0
2016-01-14 18:31:25,384 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:25,384 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:25,384 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:25,385 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:25,385 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:25,385 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:25,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: EDAF121338594169AB05786F9781F36A, VertexParallelism: 1, TaskAttemptID:attempt_1452796285143_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=30BCA951C61841E28202694F92D62A02, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D8C3760D843D4C79B4044B9739F29EF4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:25,386 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:25,386 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796285143_0001_1_00_000000_0
2016-01-14 18:31:25,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:25,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:25,388 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796285143_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:25,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=30BCA951C61841E28202694F92D62A02, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:25,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:25,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:25,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 30BCA951C61841E28202694F92D62A02
2016-01-14 18:31:25,394 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:25,394 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 30BCA951C61841E28202694F92D62A02
2016-01-14 18:31:25,398 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:25,398 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 30BCA951C61841E28202694F92D62A02
2016-01-14 18:31:25,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 30BCA951C61841E28202694F92D62A02
2016-01-14 18:31:25,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D8C3760D843D4C79B4044B9739F29EF4, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:25,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:25,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D8C3760D843D4C79B4044B9739F29EF4
2016-01-14 18:31:25,401 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EDAF121338594169AB05786F9781F36A, ordinal: 0
2016-01-14 18:31:25,401 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EDAF121338594169AB05786F9781F36A, mem on start (mb), free: 282, total: 370, max: 672
2016-01-14 18:31:25,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:25,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:25,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:25,404 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:25,404 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D8C3760D843D4C79B4044B9739F29EF4
2016-01-14 18:31:25,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D8C3760D843D4C79B4044B9739F29EF4
2016-01-14 18:31:25,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:25,405 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:25,406 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:25,407 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:25,407 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:25,408 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:25,408 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:25,408 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 30BCA951C61841E28202694F92D62A02, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:25,408 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D8C3760D843D4C79B4044B9739F29EF4, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:25,408 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:25,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 30BCA951C61841E28202694F92D62A02
2016-01-14 18:31:25,409 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:25,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 30BCA951C61841E28202694F92D62A02
2016-01-14 18:31:25,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 30BCA951C61841E28202694F92D62A02 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:25,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:25,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:25,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:25,410 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796285143_0001_1_00_000000_0
2016-01-14 18:31:25,414 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:25,414 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 30BCA951C61841E28202694F92D62A02
2016-01-14 18:31:25,414 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 9CE54D404D79447C90905BF17863FD29
2016-01-14 18:31:25,415 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 9CE54D404D79447C90905BF17863FD29
2016-01-14 18:31:25,416 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@ce59f7b; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@37494ada; comparator=cascading.tuple.hadoop.util.TupleComparator@2cd14a91; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:25,416 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:25,416 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:25,417 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:25,420 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:25,422 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@15e5b299
2016-01-14 18:31:25,424 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@15e5b299
2016-01-14 18:31:25,424 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:25,425 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:25,436 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:25,436 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:25,436 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:25,436 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:25,437 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:25,437 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:25,437 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:25,438 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EDAF121338594169AB05786F9781F36A, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:25,441 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796285143_0001_1_00_000000_0
2016-01-14 18:31:25,442 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EDAF121338594169AB05786F9781F36A, mem on close (mb), free: 179, total: 370, max: 672
2016-01-14 18:31:25,442 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:25,442 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:25,442 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:25,442 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:25,445 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:25 Completed running task attempt: attempt_1452796285143_0001_1_00_000000_0
2016-01-14 18:31:25,450 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796285143_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:25,451 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796285438, Process_Duration=3, Process_End_Time=1452796285441, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:25,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796285143_0001_1_00, eventCount=1
2016-01-14 18:31:25,451 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:25,451 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:25,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EDAF121338594169AB05786F9781F36A, taskAttemptId=attempt_1452796285143_0001_1_00_000000_0, startTime=1452796285382, finishTime=1452796285452, timeTaken=70, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796285438, Process_Duration=3, Process_End_Time=1452796285441, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:25,452 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285143_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:25,452 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:25,452 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796285143_0001_1_00_000000_0
2016-01-14 18:31:25,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:25,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:TASK_FINISHED]: vertexName=EDAF121338594169AB05786F9781F36A, taskId=task_1452796285143_0001_1_00_000000, startTime=1452796285382, finishTime=1452796285452, timeTaken=70, status=SUCCEEDED, successfulAttemptID=attempt_1452796285143_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796285438, Process_Duration=3, Process_End_Time=1452796285441, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:25,453 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285143_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:25,453 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A] : 1
2016-01-14 18:31:25,453 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A]
2016-01-14 18:31:25,452 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796285143_0001_00_000001
2016-01-14 18:31:25,453 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:25,454 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:25,454 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:25,455 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:25,455 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:VERTEX_FINISHED]: vertexName=EDAF121338594169AB05786F9781F36A, vertexId=vertex_1452796285143_0001_1_00, initRequestedTime=1452796285337, initedTime=1452796285356, startRequestedTime=1452796285358, startedTime=1452796285358, finishTime=1452796285453, timeTaken=95, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796285438, Process_Duration=3, Process_End_Time=1452796285441, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796285382, firstTasksToStart=[ task_1452796285143_0001_1_00_000000 ], lastTaskFinishTime=1452796285452, lastTasksToFinish=[ task_1452796285143_0001_1_00_000000 ], minTaskDuration=70, maxTaskDuration=70, avgTaskDuration=70.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796285143_0001_1_00_000000 ], longestDurationTasks=[ task_1452796285143_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:25,455 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:25,455 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4] attempt: attempt_1452796285143_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:25,455 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D8C3760D843D4C79B4044B9739F29EF4
2016-01-14 18:31:25,456 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4]
2016-01-14 18:31:25,457 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:25,457 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:25,457 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:25,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:TASK_STARTED]: vertexName=D8C3760D843D4C79B4044B9739F29EF4, taskId=task_1452796285143_0001_1_01_000000, scheduledTime=1452796285457, launchTime=1452796285457
2016-01-14 18:31:25,458 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285143_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:25,458 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:25,458 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796285143_0001_00_000001, stoppedTime=1452796285458, exitStatus=0
2016-01-14 18:31:25,459 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:25,459 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4]
2016-01-14 18:31:25,459 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796285143_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:25,460 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: D8C3760D843D4C79B4044B9739F29EF4, VertexParallelism: 1, TaskAttemptID:attempt_1452796285143_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EDAF121338594169AB05786F9781F36A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7EB2DCB408194CE9B6A4948DA3D6E073, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:25,460 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285143_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:25,460 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:25,461 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:25,461 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796285143_0001_00_000002
2016-01-14 18:31:25,462 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:25,464 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796285143_0001_00_000002
2016-01-14 18:31:25,464 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:25,465 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:25,465 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:25,465 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796285143_0001_00_000002, launchTime=1452796285465
2016-01-14 18:31:25,466 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796285143_0001_00_000002
2016-01-14 18:31:25,467 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796285143_0001_1_01_000000_0] to container: [container_1452796285143_0001_00_000002]
2016-01-14 18:31:25,467 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:25,467 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796285143_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:25,468 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796285143_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:25,468 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796285143_0001_1_01_000000_0] started. Is using containerId: [container_1452796285143_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:25,468 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796285143_0001_00_000001, Exiting
2016-01-14 18:31:25,468 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796285143_0001_00_000001
2016-01-14 18:31:25,468 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D8C3760D843D4C79B4044B9739F29EF4, taskAttemptId=attempt_1452796285143_0001_1_01_000000_0, startTime=1452796285467, containerId=container_1452796285143_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796285143_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:25,468 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285143_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:25,468 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796285143_0001_00_000001
2016-01-14 18:31:25,468 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:25,468 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285143_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:25,469 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796285143_0001_00_000002 given task: attempt_1452796285143_0001_1_01_000000_0
2016-01-14 18:31:25,469 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796285143_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796285143_0001_1_01_000000_0
2016-01-14 18:31:25 Starting to run new task attempt: attempt_1452796285143_0001_1_01_000000_0
2016-01-14 18:31:25,470 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796285143_0001_1_01_000000_0
2016-01-14 18:31:25,470 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:25,470 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:25,470 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:25,471 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:25,471 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:25,471 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:25,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67] every/(1/1) ...semblyplatform/-1_ARGS_-1, VertexName: D8C3760D843D4C79B4044B9739F29EF4, VertexParallelism: 1, TaskAttemptID:attempt_1452796285143_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EDAF121338594169AB05786F9781F36A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7EB2DCB408194CE9B6A4948DA3D6E073, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:25,471 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:25,472 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796285143_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:25,472 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796285143_0001_00_000001 completed successfully
2016-01-14 18:31:25,472 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:25,473 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796285143_0001_1_01_000000_0
2016-01-14 18:31:25,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:25,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:25,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EDAF121338594169AB05786F9781F36A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:25,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:25,476 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EDAF121338594169AB05786F9781F36A
2016-01-14 18:31:25,477 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:25,478 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:25,478 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EDAF121338594169AB05786F9781F36A
2016-01-14 18:31:25,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EDAF121338594169AB05786F9781F36A
2016-01-14 18:31:25,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:25,482 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:25,484 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796285143_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:25,486 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7EB2DCB408194CE9B6A4948DA3D6E073, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:25,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:25,487 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7EB2DCB408194CE9B6A4948DA3D6E073
2016-01-14 18:31:25,487 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:25,488 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7EB2DCB408194CE9B6A4948DA3D6E073
2016-01-14 18:31:25,497 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D8C3760D843D4C79B4044B9739F29EF4, ordinal: 1
2016-01-14 18:31:25,497 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8C3760D843D4C79B4044B9739F29EF4, mem on start (mb), free: 172, total: 370, max: 672
2016-01-14 18:31:25,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:25,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:25,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:25,501 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:25,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7EB2DCB408194CE9B6A4948DA3D6E073
2016-01-14 18:31:25,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:25,502 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:25,502 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:25,502 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:25,502 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:25,503 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:25,503 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:25,504 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EDAF121338594169AB05786F9781F36A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:25,504 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7EB2DCB408194CE9B6A4948DA3D6E073, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:25,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:25,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EDAF121338594169AB05786F9781F36A
2016-01-14 18:31:25,505 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:25,506 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:25,506 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EDAF121338594169AB05786F9781F36A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:25,506 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:25,506 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:25,507 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:25,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EDAF121338594169AB05786F9781F36A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:25,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EDAF121338594169AB05786F9781F36A
2016-01-14 18:31:25,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:25,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:25,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:25,513 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796285143_0001_1_01_000000_0
2016-01-14 18:31:25,513 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796285143_0001_1_00_000000_0_10093, runDuration: 47733, ]
2016-01-14 18:31:25,515 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796285143_0001_1_00_000000_0_10093] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796285515, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:25,516 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EDAF121338594169AB05786F9781F36A
2016-01-14 18:31:25,516 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:25,516 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [EDAF121338594169AB05786F9781F36A] #16 in 2ms
2016-01-14 18:31:25,517 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:25,518 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EDAF121338594169AB05786F9781F36A] #16, status:false, isInterrupted:false
2016-01-14 18:31:25,518 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 9CE54D404D79447C90905BF17863FD29
2016-01-14 18:31:25,518 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:25,519 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1"], id: 7EB2DCB408194CE9B6A4948DA3D6E073
2016-01-14 18:31:25,519 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_-1"]
2016-01-14 18:31:25,519 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 9CE54D404D79447C90905BF17863FD29, for 1 inputs
2016-01-14 18:31:25,520 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:25,520 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:25,520 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:25,521 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:25,521 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EDAF121338594169AB05786F9781F36A
2016-01-14 18:31:25,521 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:25,521 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D8C3760D843D4C79B4044B9739F29EF4, all 1 inputs ready in: 00:00:00.002
2016-01-14 18:31:25,522 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@70973d9b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@594dc4dc
2016-01-14 18:31:25,527 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:25,527 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:25,527 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796285143_0001_1_01_000000_0
2016-01-14 18:31:25,528 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796285143_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:25,528 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796285143_0001_1_01_000000_0
2016-01-14 18:31:25,529 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8C3760D843D4C79B4044B9739F29EF4, mem on close (mb), free: 166, total: 370, max: 672
2016-01-14 18:31:25,529 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EDAF121338594169AB05786F9781F36A
2016-01-14 18:31:25,529 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:25,530 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:25,530 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:25,530 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:25,530 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:25 Completed running task attempt: attempt_1452796285143_0001_1_01_000000_0
2016-01-14 18:31:25,531 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796285143_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:25,533 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D8C3760D843D4C79B4044B9739F29EF4, taskAttemptId=attempt_1452796285143_0001_1_01_000000_0, startTime=1452796285467, finishTime=1452796285532, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796285522, Process_Duration=6, Process_End_Time=1452796285528, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:25,533 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285143_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:25,533 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796285143_0001_1_01_000000_0
2016-01-14 18:31:25,533 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:25,534 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:25,535 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:25,534 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:TASK_FINISHED]: vertexName=D8C3760D843D4C79B4044B9739F29EF4, taskId=task_1452796285143_0001_1_01_000000, startTime=1452796285467, finishTime=1452796285533, timeTaken=66, status=SUCCEEDED, successfulAttemptID=attempt_1452796285143_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796285522, Process_Duration=6, Process_End_Time=1452796285528, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:25,536 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285143_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:25,536 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4] : 1
2016-01-14 18:31:25,536 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4]
2016-01-14 18:31:25,537 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:25,537 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:VERTEX_FINISHED]: vertexName=D8C3760D843D4C79B4044B9739F29EF4, vertexId=vertex_1452796285143_0001_1_01, initRequestedTime=1452796285338, initedTime=1452796285349, startRequestedTime=1452796285361, startedTime=1452796285361, finishTime=1452796285536, timeTaken=175, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796285522, Process_Duration=6, Process_End_Time=1452796285528, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796285467, firstTasksToStart=[ task_1452796285143_0001_1_01_000000 ], lastTaskFinishTime=1452796285532, lastTasksToFinish=[ task_1452796285143_0001_1_01_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796285143_0001_1_01_000000 ], longestDurationTasks=[ task_1452796285143_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:25,537 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:25,538 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:25,538 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796285143_0001_00_000002, stoppedTime=1452796285538, exitStatus=0
2016-01-14 18:31:25,538 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:25,538 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:25,539 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:25,539 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796285143_0001_1
2016-01-14 18:31:25,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796285143_0001_1
2016-01-14 18:31:25,539 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796285143_0001_1_00 [EDAF121338594169AB05786F9781F36A]
2016-01-14 18:31:25,540 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 7EB2DCB408194CE9B6A4948DA3D6E073 for vertex: vertex_1452796285143_0001_1_01 [D8C3760D843D4C79B4044B9739F29EF4]
2016-01-14 18:31:25,543 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796285522, Process_Duration=6, Process_End_Time=1452796285528, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:25,544 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:25,544 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:25,544 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796285143_0001_00_000002
2016-01-14 18:31:25,545 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796285143_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:25,545 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796285143_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:25,545 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796285143_0001_00_000002, Exiting
2016-01-14 18:31:25,545 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796285143_0001_00_000002
2016-01-14 18:31:25,545 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796285143_0001_00_000002
2016-01-14 18:31:25,546 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796285143_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:25,546 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796285143_0001_00_000002 completed successfully
2016-01-14 18:31:25,551 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796285143_0001_1, queueSize=0
2016-01-14 18:31:25,551 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:25,551 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285143_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796285143_0001_1, startTime=1452796285336, finishTime=1452796285549, timeTaken=213, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=776994816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592570960, Process_Duration=9, Process_End_Time=2905592570969, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:25,551 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796285143_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:25,551 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796285143_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:25,551 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285143_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:25,552 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796285143_0001_00_000002
2016-01-14 18:31:25 Completed Dag: dag_1452796285143_0001_1
2016-01-14 18:31:25,552 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796285143_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:25,552 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796285143_0001_1_post
2016-01-14 18:31:25,552 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:25,552 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:25,559 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...semblyplatform/-1_ARGS_-1, applicationId=application_1452796285143_0001
2016-01-14 18:31:25,561 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:25,561 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:25,561 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:25,561 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:25,562 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:25,562 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:25,562 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:25,563 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:25,563 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:25,563 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:25,564 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:25,564 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:25,564 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:25,570 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D0FE8DA691F346CE88E25675A037AC53/E8809C5E436F4201BB25E13701DD5C67/.tez/application_1452796285143_0001
2016-01-14 18:31:25,571 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.460
2016-01-14 18:31:25,642 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_-1
2016-01-14 18:31:25,667 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:25,667 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:25,710 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-01-14 18:31:25,710 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-01-14 18:31:25,711 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:25,711 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:25,712 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_'field'
2016-01-14 18:31:25,752 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:25,753 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:25,795 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 18:31:25,797 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 18:31:25,798 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:25,799 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:25,799 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:25,824 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:25,834 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS
2016-01-14 18:31:25,897 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:25,897 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:25,897 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS"]
2016-01-14 18:31:25,898 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:25,898 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:25,898 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:25,899 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/-1_ARGS_RESULTS
2016-01-14 18:31:25,899 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:25,918 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:25,922 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:25,922 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:25,922 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:25,930 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573/.tez/application_1452796285922_0001 doesn't exist and is created
2016-01-14 18:31:25,944 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:25,944 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:25,945 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573/.tez/application_1452796285922_0001
2016-01-14 18:31:25,947 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796285922_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:25,957 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:25,958 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:25,959 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:25,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:25,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573/.tez/application_1452796285922_0001/localmode-log-dir
2016-01-14 18:31:25,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573/.tez/application_1452796285922_0001/localmode-log-dir/history.txt.appattempt_1452796285922_0001_000000, maxErrors=10
2016-01-14 18:31:25,960 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:25,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796285922_0001_000000, appSubmitTime=1452796285946, launchTime=1452796285946
2016-01-14 18:31:25,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:25,966 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:25,967 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:25,967 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:25,967 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:34713
2016-01-14 18:31:25,971 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:25,973 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796285922_0001_000000, startTime=1452796285973
2016-01-14 18:31:25,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:25,973 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:26,044 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:26,045 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:26,045 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1452796285922_0001, dagName=[993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-01-14 18:31:26,063 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-01-14 18:31:26,065 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796285922_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573/.tez/application_1452796285922_0001/localmode-log-dir/dag_1452796285922_0001_1.dot
2016-01-14 18:31:26,065 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573/.tez/application_1452796285922_0001/localmode-log-dir/dag_1452796285922_0001_1-tez-dag.pb.txt
2016-01-14 18:31:26,076 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796285922_0001_1
2016-01-14 18:31:26,076 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,076 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-01-14 18:31:26 Running Dag: dag_1452796285922_0001_1
2016-01-14 18:31:26,086 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796285922_0001_1, submitTime=1452796286063
2016-01-14 18:31:26,086 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:26,087 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B4A59FD55980474291DE01831CC96359
2016-01-14 18:31:26,088 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:26,088 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:26,089 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796285922_0001_1, initTime=1452796286086
2016-01-14 18:31:26,090 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796285922_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:26,091 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:DAG_STARTED]: dagID=dag_1452796285922_0001_1, startTime=1452796286091
2016-01-14 18:31:26,091 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:26,091 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:26,091 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796285922_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:26,092 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1452796285922_0001, dagName=[993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573] every/(1/1) ...yplatform/-1_ARGS_RESULTS
2016-01-14 18:31:26,092 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DA796D5160B24A5A82231169808839BA : {5B07A1A6B6174FF38CB0F27F54ABE72E={InputName=5B07A1A6B6174FF38CB0F27F54ABE72E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:26,092 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 5B07A1A6B6174FF38CB0F27F54ABE72E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:26,092 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA]
2016-01-14 18:31:26,092 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA]
2016-01-14 18:31:26,092 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA]
2016-01-14 18:31:26,093 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA]. Starting root input initializers: 1
2016-01-14 18:31:26,093 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:26,093 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359]
2016-01-14 18:31:26,094 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 5B07A1A6B6174FF38CB0F27F54ABE72E on vertex vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA]
2016-01-14 18:31:26,096 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:26,097 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:26,097 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359]
2016-01-14 18:31:26,097 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359]
2016-01-14 18:31:26,097 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359]
2016-01-14 18:31:26,097 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=EE8FFC91F0904676BCB0E0FEEFD103ED, vertexId=vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:26,097 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=EE8FFC91F0904676BCB0E0FEEFD103ED, vertexId=vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359]
2016-01-14 18:31:26,097 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 5B07A1A6B6174FF38CB0F27F54ABE72E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:26,099 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:26,099 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:26,099 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=EE8FFC91F0904676BCB0E0FEEFD103ED, vertexName=B4A59FD55980474291DE01831CC96359, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:26,100 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=EE8FFC91F0904676BCB0E0FEEFD103ED, vertexId=vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359]
2016-01-14 18:31:26,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:26,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B4A59FD55980474291DE01831CC96359, vertexId=vertex_1452796285922_0001_1_01, initRequestedTime=1452796286093, initedTime=1452796286101, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:26,101 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:26,102 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:26,102 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 5B07A1A6B6174FF38CB0F27F54ABE72E on vertex vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA]
2016-01-14 18:31:26,102 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:26,102 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {5B07A1A6B6174FF38CB0F27F54ABE72E=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:26,102 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA] parallelism set to 1
2016-01-14 18:31:26,102 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:26,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796285922_0001_1_00, eventCount=1
2016-01-14 18:31:26,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DA796D5160B24A5A82231169808839BA, vertexId=vertex_1452796285922_0001_1_00, initRequestedTime=1452796286092, initedTime=1452796286103, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:26,103 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA] with distanceFromRoot: 0
2016-01-14 18:31:26,103 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:26,104 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:26,104 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DA796D5160B24A5A82231169808839BA
2016-01-14 18:31:26,104 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA]
2016-01-14 18:31:26,104 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796285922_0001_1_00, startRequestedTime=1452796286104, startedTime=1452796286104
2016-01-14 18:31:26,104 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:26,104 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:26,105 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:26,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:TASK_STARTED]: vertexName=DA796D5160B24A5A82231169808839BA, taskId=task_1452796285922_0001_1_00_000000, scheduledTime=1452796286105, launchTime=1452796286105
2016-01-14 18:31:26,105 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285922_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:26,105 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796285922_0001_1_00 for vertex: vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359] numStartedSources: 1 numSources: 1
2016-01-14 18:31:26,106 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359] with distanceFromRoot: 1
2016-01-14 18:31:26,106 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA]
2016-01-14 18:31:26,106 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: DA796D5160B24A5A82231169808839BA in vertex: B4A59FD55980474291DE01831CC96359
2016-01-14 18:31:26,106 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B4A59FD55980474291DE01831CC96359 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:26,107 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796285922_0001_1_01, startRequestedTime=1452796286106, startedTime=1452796286106
2016-01-14 18:31:26,107 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:26,107 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:26,107 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: DA796D5160B24A5A82231169808839BA, VertexParallelism: 1, TaskAttemptID:attempt_1452796285922_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5B07A1A6B6174FF38CB0F27F54ABE72E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B4A59FD55980474291DE01831CC96359, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:26,108 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285922_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:26,108 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:26,108 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:26,109 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:26,109 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:26,109 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796285922_0001_00_000001
2016-01-14 18:31:26,109 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:26,111 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796285922_0001_00_000001
2016-01-14 18:31:26,111 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:26,112 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:26,112 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:26,113 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796285922_0001_00_000001, launchTime=1452796286112
2016-01-14 18:31:26,113 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796285922_0001_00_000001
2016-01-14 18:31:26,114 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796285922_0001_1_00_000000_0] to container: [container_1452796285922_0001_00_000001]
2016-01-14 18:31:26,114 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:26,114 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796285922_0001_00_000001 given task: attempt_1452796285922_0001_1_00_000000_0
2016-01-14 18:31:26,114 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796285922_0001_1_00_000000_0] started. Is using containerId: [container_1452796285922_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:26,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DA796D5160B24A5A82231169808839BA, taskAttemptId=attempt_1452796285922_0001_1_00_000000_0, startTime=1452796286114, containerId=container_1452796285922_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796285922_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:26,115 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285922_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:26,116 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285922_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:26,117 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796285922_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796285922_0001_1_00_000000_0
2016-01-14 18:31:26 Starting to run new task attempt: attempt_1452796285922_0001_1_00_000000_0
2016-01-14 18:31:26,118 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796285922_0001, with dag id: dag_1452796285922_0001_1
2016-01-14 18:31:26,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:26,119 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796285922_0001_1_00_000000_0
2016-01-14 18:31:26,120 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,120 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:26,120 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:26,120 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:26,120 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:26,121 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:26,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: DA796D5160B24A5A82231169808839BA, VertexParallelism: 1, TaskAttemptID:attempt_1452796285922_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=5B07A1A6B6174FF38CB0F27F54ABE72E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B4A59FD55980474291DE01831CC96359, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:26,121 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:26,123 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796285922_0001_1_00_000000_0
2016-01-14 18:31:26,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:26,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,124 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796285922_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:26,126 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B4A59FD55980474291DE01831CC96359, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:26,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=5B07A1A6B6174FF38CB0F27F54ABE72E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:26,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:26,128 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 5B07A1A6B6174FF38CB0F27F54ABE72E
2016-01-14 18:31:26,128 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 5B07A1A6B6174FF38CB0F27F54ABE72E
2016-01-14 18:31:26,129 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:26,131 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:26,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:26,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B4A59FD55980474291DE01831CC96359
2016-01-14 18:31:26,131 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 5B07A1A6B6174FF38CB0F27F54ABE72E
2016-01-14 18:31:26,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 5B07A1A6B6174FF38CB0F27F54ABE72E
2016-01-14 18:31:26,133 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:26,134 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B4A59FD55980474291DE01831CC96359
2016-01-14 18:31:26,134 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B4A59FD55980474291DE01831CC96359
2016-01-14 18:31:26,135 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DA796D5160B24A5A82231169808839BA, ordinal: 0
2016-01-14 18:31:26,135 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA796D5160B24A5A82231169808839BA, mem on start (mb), free: 246, total: 371, max: 672
2016-01-14 18:31:26,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:26,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:26,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:26,136 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:26,136 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:26,137 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:26,137 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:26,137 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:26,137 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:26,138 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 5B07A1A6B6174FF38CB0F27F54ABE72E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:26,138 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B4A59FD55980474291DE01831CC96359, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:26,138 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:26,138 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 5B07A1A6B6174FF38CB0F27F54ABE72E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:26,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:26,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:26,138 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 5B07A1A6B6174FF38CB0F27F54ABE72E
2016-01-14 18:31:26,139 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:26,139 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 5B07A1A6B6174FF38CB0F27F54ABE72E
2016-01-14 18:31:26,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:26,140 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796285922_0001_1_00_000000_0
2016-01-14 18:31:26,140 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:26,141 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7dd9d603
2016-01-14 18:31:26,143 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7dd9d603
2016-01-14 18:31:26,143 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:26,143 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:26,144 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:26,145 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 5B07A1A6B6174FF38CB0F27F54ABE72E
2016-01-14 18:31:26,145 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 9C113E7F66D143E4BB07E5FE45891337
2016-01-14 18:31:26,145 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 9C113E7F66D143E4BB07E5FE45891337
2016-01-14 18:31:26,146 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@4ec6ff50; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6aa40597; comparator=cascading.tuple.hadoop.util.TupleComparator@366b3333; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:26,146 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:26,146 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:26,146 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:26,159 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:26,160 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:26,160 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:26,160 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:26,160 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:26,161 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:26,161 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:26,161 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DA796D5160B24A5A82231169808839BA, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:26,164 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796285922_0001_1_00_000000_0
2016-01-14 18:31:26,164 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DA796D5160B24A5A82231169808839BA, mem on close (mb), free: 144, total: 371, max: 672
2016-01-14 18:31:26,164 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:26,165 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:26,165 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:26,165 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:26,167 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:26 Completed running task attempt: attempt_1452796285922_0001_1_00_000000_0
2016-01-14 18:31:26,172 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796285922_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:26,173 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:26,173 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:26,175 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796285922_0001_1_00, eventCount=1
2016-01-14 18:31:26,175 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DA796D5160B24A5A82231169808839BA, taskAttemptId=attempt_1452796285922_0001_1_00_000000_0, startTime=1452796286114, finishTime=1452796286175, timeTaken=61, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286162, Process_Duration=2, Process_End_Time=1452796286164, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:26,175 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285922_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:26,175 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:26,176 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796285922_0001_1_00_000000_0
2016-01-14 18:31:26,176 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:26,176 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:TASK_FINISHED]: vertexName=DA796D5160B24A5A82231169808839BA, taskId=task_1452796285922_0001_1_00_000000, startTime=1452796286114, finishTime=1452796286176, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1452796285922_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286162, Process_Duration=2, Process_End_Time=1452796286164, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:26,176 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285922_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:26,176 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA] : 1
2016-01-14 18:31:26,177 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA]
2016-01-14 18:31:26,177 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:26,178 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:26,179 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:VERTEX_FINISHED]: vertexName=DA796D5160B24A5A82231169808839BA, vertexId=vertex_1452796285922_0001_1_00, initRequestedTime=1452796286092, initedTime=1452796286103, startRequestedTime=1452796286104, startedTime=1452796286104, finishTime=1452796286177, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286162, Process_Duration=2, Process_End_Time=1452796286164, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796286114, firstTasksToStart=[ task_1452796285922_0001_1_00_000000 ], lastTaskFinishTime=1452796286175, lastTasksToFinish=[ task_1452796285922_0001_1_00_000000 ], minTaskDuration=61, maxTaskDuration=61, avgTaskDuration=61.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796285922_0001_1_00_000000 ], longestDurationTasks=[ task_1452796285922_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:26,179 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:26,179 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359] attempt: attempt_1452796285922_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:26,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796286162, Process_Duration=2, Process_End_Time=1452796286164, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:26,179 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:26,180 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,180 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B4A59FD55980474291DE01831CC96359
2016-01-14 18:31:26,180 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359]
2016-01-14 18:31:26,180 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:26,180 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796285922_0001_00_000001
2016-01-14 18:31:26,180 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:26,181 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796285922_0001_00_000001, stoppedTime=1452796286181, exitStatus=0
2016-01-14 18:31:26,181 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:26,181 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:26,181 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:26,181 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:TASK_STARTED]: vertexName=B4A59FD55980474291DE01831CC96359, taskId=task_1452796285922_0001_1_01_000000, scheduledTime=1452796286181, launchTime=1452796286181
2016-01-14 18:31:26,182 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285922_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:26,182 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796285922_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:26,182 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359]
2016-01-14 18:31:26,183 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:26,183 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: B4A59FD55980474291DE01831CC96359, VertexParallelism: 1, TaskAttemptID:attempt_1452796285922_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DA796D5160B24A5A82231169808839BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EE8FFC91F0904676BCB0E0FEEFD103ED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:26,183 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285922_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:26,184 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:26,186 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796285922_0001_00_000002
2016-01-14 18:31:26,187 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:26,189 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796285922_0001_00_000002
2016-01-14 18:31:26,189 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:26,191 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796285922_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:26,191 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796285922_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:26,191 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796285922_0001_00_000001, Exiting
2016-01-14 18:31:26,191 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796285922_0001_00_000001
2016-01-14 18:31:26,191 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796285922_0001_00_000001
2016-01-14 18:31:26,191 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:26,192 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:26,192 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796285922_0001_00_000002, launchTime=1452796286192
2016-01-14 18:31:26,193 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796285922_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:26,193 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796285922_0001_00_000001 completed successfully
2016-01-14 18:31:26,193 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:26,194 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796285922_0001_00_000002
2016-01-14 18:31:26,194 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796285922_0001_1_01_000000_0] to container: [container_1452796285922_0001_00_000002]
2016-01-14 18:31:26,195 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:26,195 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796285922_0001_1_01_000000_0] started. Is using containerId: [container_1452796285922_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:26,195 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B4A59FD55980474291DE01831CC96359, taskAttemptId=attempt_1452796285922_0001_1_01_000000_0, startTime=1452796286195, containerId=container_1452796285922_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796285922_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:26,196 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285922_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:26,196 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285922_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:26,196 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:26,196 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796285922_0001_00_000002 given task: attempt_1452796285922_0001_1_01_000000_0
2016-01-14 18:31:26,196 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796285922_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796285922_0001_1_01_000000_0
2016-01-14 18:31:26 Starting to run new task attempt: attempt_1452796285922_0001_1_01_000000_0
2016-01-14 18:31:26,197 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796285922_0001_1_01_000000_0
2016-01-14 18:31:26,197 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,197 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:26,197 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:26,197 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:26,197 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:26,198 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:26,198 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573] every/(1/1) ...yplatform/-1_ARGS_RESULTS, VertexName: B4A59FD55980474291DE01831CC96359, VertexParallelism: 1, TaskAttemptID:attempt_1452796285922_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DA796D5160B24A5A82231169808839BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EE8FFC91F0904676BCB0E0FEEFD103ED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:26,198 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:26,200 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796285922_0001_1_01_000000_0
2016-01-14 18:31:26,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:26,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,201 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796285922_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:26,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DA796D5160B24A5A82231169808839BA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:26,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:26,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DA796D5160B24A5A82231169808839BA
2016-01-14 18:31:26,204 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:26,204 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:26,205 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: DA796D5160B24A5A82231169808839BA
2016-01-14 18:31:26,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DA796D5160B24A5A82231169808839BA
2016-01-14 18:31:26,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EE8FFC91F0904676BCB0E0FEEFD103ED, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:26,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:26,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EE8FFC91F0904676BCB0E0FEEFD103ED
2016-01-14 18:31:26,209 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:26,210 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: EE8FFC91F0904676BCB0E0FEEFD103ED
2016-01-14 18:31:26,212 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:26,220 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B4A59FD55980474291DE01831CC96359, ordinal: 1
2016-01-14 18:31:26,220 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B4A59FD55980474291DE01831CC96359, mem on start (mb), free: 137, total: 371, max: 672
2016-01-14 18:31:26,220 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,221 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:26,221 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:26,227 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:26,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EE8FFC91F0904676BCB0E0FEEFD103ED
2016-01-14 18:31:26,228 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:26,228 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:26,228 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:26,228 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:26,228 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:26,228 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:26,229 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:26,229 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DA796D5160B24A5A82231169808839BA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:26,229 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EE8FFC91F0904676BCB0E0FEEFD103ED, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:26,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:26,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DA796D5160B24A5A82231169808839BA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:26,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:26,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:26,229 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DA796D5160B24A5A82231169808839BA
2016-01-14 18:31:26,230 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:26,230 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:26,231 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: DA796D5160B24A5A82231169808839BA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:26,231 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:26,231 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:26,231 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:26,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DA796D5160B24A5A82231169808839BA
2016-01-14 18:31:26,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:26,232 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796285922_0001_1_01_000000_0
2016-01-14 18:31:26,238 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:26,239 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 9C113E7F66D143E4BB07E5FE45891337
2016-01-14 18:31:26,239 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS"], id: EE8FFC91F0904676BCB0E0FEEFD103ED
2016-01-14 18:31:26,239 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_RESULTS"]
2016-01-14 18:31:26,239 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 9C113E7F66D143E4BB07E5FE45891337, for 1 inputs
2016-01-14 18:31:26,240 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796285922_0001_1_00_000000_0_10099, runDuration: 37912, ]
2016-01-14 18:31:26,242 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796285922_0001_1_00_000000_0_10099] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796286242, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:26,242 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : DA796D5160B24A5A82231169808839BA
2016-01-14 18:31:26,242 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:26,242 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [DA796D5160B24A5A82231169808839BA] #17 in 1ms
2016-01-14 18:31:26,243 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [DA796D5160B24A5A82231169808839BA] #17, status:false, isInterrupted:false
2016-01-14 18:31:26,243 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:26,244 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:26,244 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:26,244 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:26,245 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:26,245 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B4A59FD55980474291DE01831CC96359, all 1 inputs ready in: 00:00:00.006
2016-01-14 18:31:26,245 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : DA796D5160B24A5A82231169808839BA
2016-01-14 18:31:26,246 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6f034e0e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@41f9b212
2016-01-14 18:31:26,247 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:26,250 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:26,250 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:26,251 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796285922_0001_1_01_000000_0
2016-01-14 18:31:26,252 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796285922_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:26,253 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796285922_0001_1_01_000000_0
2016-01-14 18:31:26,253 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B4A59FD55980474291DE01831CC96359, mem on close (mb), free: 132, total: 371, max: 672
2016-01-14 18:31:26,253 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: DA796D5160B24A5A82231169808839BA
2016-01-14 18:31:26,255 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:26,255 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:26,256 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:26,256 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:26,257 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:26 Completed running task attempt: attempt_1452796285922_0001_1_01_000000_0
2016-01-14 18:31:26,258 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796285922_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:26,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796286245, Process_Duration=8, Process_End_Time=1452796286253, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:26,261 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B4A59FD55980474291DE01831CC96359, taskAttemptId=attempt_1452796285922_0001_1_01_000000_0, startTime=1452796286195, finishTime=1452796286260, timeTaken=65, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286245, Process_Duration=8, Process_End_Time=1452796286253, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:26,261 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:26,261 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796285922_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:26,261 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,261 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:26,261 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796285922_0001_00_000002
2016-01-14 18:31:26,261 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:26,261 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796285922_0001_1_01_000000_0
2016-01-14 18:31:26,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:TASK_FINISHED]: vertexName=B4A59FD55980474291DE01831CC96359, taskId=task_1452796285922_0001_1_01_000000, startTime=1452796286195, finishTime=1452796286262, timeTaken=67, status=SUCCEEDED, successfulAttemptID=attempt_1452796285922_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286245, Process_Duration=8, Process_End_Time=1452796286253, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:26,263 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796285922_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:26,263 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:26,264 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359] : 1
2016-01-14 18:31:26,264 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359]
2016-01-14 18:31:26,266 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:26,266 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:VERTEX_FINISHED]: vertexName=B4A59FD55980474291DE01831CC96359, vertexId=vertex_1452796285922_0001_1_01, initRequestedTime=1452796286093, initedTime=1452796286101, startRequestedTime=1452796286106, startedTime=1452796286106, finishTime=1452796286264, timeTaken=158, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286245, Process_Duration=8, Process_End_Time=1452796286253, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796286195, firstTasksToStart=[ task_1452796285922_0001_1_01_000000 ], lastTaskFinishTime=1452796286260, lastTasksToFinish=[ task_1452796285922_0001_1_01_000000 ], minTaskDuration=65, maxTaskDuration=65, avgTaskDuration=65.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796285922_0001_1_01_000000 ], longestDurationTasks=[ task_1452796285922_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:26,267 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:26,267 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:26,267 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796285922_0001_00_000002, stoppedTime=1452796286267, exitStatus=0
2016-01-14 18:31:26,267 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:26,267 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:26,267 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796285922_0001_1
2016-01-14 18:31:26,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796285922_0001_1
2016-01-14 18:31:26,268 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796285922_0001_1_00 [DA796D5160B24A5A82231169808839BA]
2016-01-14 18:31:26,268 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: EE8FFC91F0904676BCB0E0FEEFD103ED for vertex: vertex_1452796285922_0001_1_01 [B4A59FD55980474291DE01831CC96359]
2016-01-14 18:31:26,270 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:26,273 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796285922_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:26,273 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796285922_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:26,274 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796285922_0001_00_000002, Exiting
2016-01-14 18:31:26,274 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796285922_0001_00_000002
2016-01-14 18:31:26,274 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796285922_0001_00_000002
2016-01-14 18:31:26,275 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796285922_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:26,275 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796285922_0001_00_000002 completed successfully
2016-01-14 18:31:26,278 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796285922_0001_1, queueSize=0
2016-01-14 18:31:26,279 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796285922_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796285922_0001_1, startTime=1452796286091, finishTime=1452796286276, timeTaken=185, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=778043392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=13, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=11, LAST_EVENT_RECEIVED=11, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592572407, Process_Duration=10, Process_End_Time=2905592572417, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:26,279 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796285922_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:26,279 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796285922_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:26,279 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:26,280 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796285922_0001_00_000002
2016-01-14 18:31:26,280 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796285922_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:26 Completed Dag: dag_1452796285922_0001_1
2016-01-14 18:31:26,281 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796285922_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:26,281 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796285922_0001_1_post
2016-01-14 18:31:26,281 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,281 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:26,286 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_ARGS_RESULTS, applicationId=application_1452796285922_0001
2016-01-14 18:31:26,289 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:26,289 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:26,289 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:26,289 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:26,289 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:26,292 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:26,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:26,292 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:26,292 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:26,292 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:26,293 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:26,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:26,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:26,298 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/993A1813599340F58461B56E6DFE3B7E/BA09603A38FB422CA7B9F9D585634573/.tez/application_1452796285922_0001
2016-01-14 18:31:26,298 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.401
2016-01-14 18:31:26,380 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_RESULTS
2016-01-14 18:31:26,403 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:26,404 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:26,433 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-01-14 18:31:26,434 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-01-14 18:31:26,434 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:26,434 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:26,435 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_ALL
2016-01-14 18:31:26,461 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:26,461 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:26,501 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 18:31:26,504 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 18:31:26,505 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:26,505 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:26,505 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:26,521 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:26,527 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE
2016-01-14 18:31:26,575 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:26,576 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:26,576 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE"]
2016-01-14 18:31:26,576 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:26,576 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:26,576 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:26,577 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/-1_ARGS_REPLACE
2016-01-14 18:31:26,578 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:26,592 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:26,596 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:26,596 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:26,596 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:26,605 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE/.tez/application_1452796286596_0001 doesn't exist and is created
2016-01-14 18:31:26,619 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:26,619 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:26,619 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE/.tez/application_1452796286596_0001
2016-01-14 18:31:26,621 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796286596_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:26,632 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:26,632 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:26,634 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:26,634 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:26,634 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE/.tez/application_1452796286596_0001/localmode-log-dir
2016-01-14 18:31:26,635 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE/.tez/application_1452796286596_0001/localmode-log-dir/history.txt.appattempt_1452796286596_0001_000000, maxErrors=10
2016-01-14 18:31:26,635 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:26,635 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796286596_0001_000000, appSubmitTime=1452796286621, launchTime=1452796286621
2016-01-14 18:31:26,638 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:26,641 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:26,641 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:26,642 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:26,642 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:51515
2016-01-14 18:31:26,645 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:26,647 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:26,647 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796286596_0001_000000, startTime=1452796286647
2016-01-14 18:31:26,647 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:26,719 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:26,719 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:26,719 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1452796286596_0001, dagName=[D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-01-14 18:31:26,736 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-01-14 18:31:26,737 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796286596_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE/.tez/application_1452796286596_0001/localmode-log-dir/dag_1452796286596_0001_1.dot
2016-01-14 18:31:26,738 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE/.tez/application_1452796286596_0001/localmode-log-dir/dag_1452796286596_0001_1-tez-dag.pb.txt
2016-01-14 18:31:26,744 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796286596_0001_1
2016-01-14 18:31:26,744 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,745 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-01-14 18:31:26 Running Dag: dag_1452796286596_0001_1
2016-01-14 18:31:26,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796286596_0001_1, submitTime=1452796286736
2016-01-14 18:31:26,759 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:26,760 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F59F746641C04310AC1FDC9A779671D1
2016-01-14 18:31:26,761 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:26,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796286596_0001_1, initTime=1452796286760
2016-01-14 18:31:26,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:26,761 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796286596_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:26,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:DAG_STARTED]: dagID=dag_1452796286596_0001_1, startTime=1452796286762
2016-01-14 18:31:26,762 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:26,762 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:26,762 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796286596_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:26,762 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B762F9E0E0FA4DD9994B42D2202A7450 : {2EE599F5B484450DB575B247641CBE37={InputName=2EE599F5B484450DB575B247641CBE37}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:26,763 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2EE599F5B484450DB575B247641CBE37, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:26,763 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450]
2016-01-14 18:31:26,763 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450]
2016-01-14 18:31:26,763 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450]
2016-01-14 18:31:26,763 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450]. Starting root input initializers: 1
2016-01-14 18:31:26,765 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:26,765 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1]
2016-01-14 18:31:26,765 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2EE599F5B484450DB575B247641CBE37 on vertex vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450]
2016-01-14 18:31:26,765 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1452796286596_0001, dagName=[D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE] every/(1/1) ...yplatform/-1_ARGS_REPLACE
2016-01-14 18:31:26,768 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:26,768 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1]
2016-01-14 18:31:26,768 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1]
2016-01-14 18:31:26,768 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1]
2016-01-14 18:31:26,768 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6B7650DC90D14E3BAC6318ADDC6B22C9, vertexId=vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:26,769 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6B7650DC90D14E3BAC6318ADDC6B22C9, vertexId=vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1]
2016-01-14 18:31:26,770 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6B7650DC90D14E3BAC6318ADDC6B22C9, vertexName=F59F746641C04310AC1FDC9A779671D1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:26,770 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6B7650DC90D14E3BAC6318ADDC6B22C9, vertexId=vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1]
2016-01-14 18:31:26,770 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:26,771 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2EE599F5B484450DB575B247641CBE37 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:26,771 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:26,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F59F746641C04310AC1FDC9A779671D1, vertexId=vertex_1452796286596_0001_1_01, initRequestedTime=1452796286765, initedTime=1452796286771, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:26,771 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:26,772 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:26,772 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:26,773 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:26,774 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2EE599F5B484450DB575B247641CBE37 on vertex vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450]
2016-01-14 18:31:26,774 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:26,774 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2EE599F5B484450DB575B247641CBE37=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:26,774 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450] parallelism set to 1
2016-01-14 18:31:26,774 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:26,775 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796286596_0001_1_00, eventCount=1
2016-01-14 18:31:26,775 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B762F9E0E0FA4DD9994B42D2202A7450, vertexId=vertex_1452796286596_0001_1_00, initRequestedTime=1452796286762, initedTime=1452796286775, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:26,775 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:26,775 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450] with distanceFromRoot: 0
2016-01-14 18:31:26,776 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:26,776 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B762F9E0E0FA4DD9994B42D2202A7450
2016-01-14 18:31:26,776 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450]
2016-01-14 18:31:26,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796286596_0001_1_00, startRequestedTime=1452796286776, startedTime=1452796286776
2016-01-14 18:31:26,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:26,777 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:26,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:26,777 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:TASK_STARTED]: vertexName=B762F9E0E0FA4DD9994B42D2202A7450, taskId=task_1452796286596_0001_1_00_000000, scheduledTime=1452796286777, launchTime=1452796286777
2016-01-14 18:31:26,778 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796286596_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:26,778 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796286596_0001_1_00 for vertex: vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1] numStartedSources: 1 numSources: 1
2016-01-14 18:31:26,778 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1] with distanceFromRoot: 1
2016-01-14 18:31:26,778 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450]
2016-01-14 18:31:26,779 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B762F9E0E0FA4DD9994B42D2202A7450 in vertex: F59F746641C04310AC1FDC9A779671D1
2016-01-14 18:31:26,779 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F59F746641C04310AC1FDC9A779671D1 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:26,779 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796286596_0001_1_01, startRequestedTime=1452796286779, startedTime=1452796286779
2016-01-14 18:31:26,779 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:26,779 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:26,780 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: B762F9E0E0FA4DD9994B42D2202A7450, VertexParallelism: 1, TaskAttemptID:attempt_1452796286596_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2EE599F5B484450DB575B247641CBE37, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F59F746641C04310AC1FDC9A779671D1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:26,780 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796286596_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:26,780 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:26,781 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:26,781 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:26,782 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:26,782 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796286596_0001_00_000001
2016-01-14 18:31:26,782 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:26,783 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796286596_0001_00_000001
2016-01-14 18:31:26,783 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:26,784 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:26,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:26,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796286596_0001_00_000001, launchTime=1452796286784
2016-01-14 18:31:26,785 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796286596_0001_00_000001
2016-01-14 18:31:26,786 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796286596_0001_1_00_000000_0] to container: [container_1452796286596_0001_00_000001]
2016-01-14 18:31:26,786 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:26,786 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796286596_0001_1_00_000000_0] started. Is using containerId: [container_1452796286596_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:26,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:26,786 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796286596_0001_00_000001 given task: attempt_1452796286596_0001_1_00_000000_0
2016-01-14 18:31:26,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B762F9E0E0FA4DD9994B42D2202A7450, taskAttemptId=attempt_1452796286596_0001_1_00_000000_0, startTime=1452796286786, containerId=container_1452796286596_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796286596_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:26,787 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796286596_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:26,787 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796286596_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:26,787 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796286596_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796286596_0001_1_00_000000_0
2016-01-14 18:31:26 Starting to run new task attempt: attempt_1452796286596_0001_1_00_000000_0
2016-01-14 18:31:26,787 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796286596_0001_1_00_000000_0
2016-01-14 18:31:26,788 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,787 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796286596_0001, with dag id: dag_1452796286596_0001_1
2016-01-14 18:31:26,788 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:26,788 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:26,788 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:26,789 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:26,789 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:26,789 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: B762F9E0E0FA4DD9994B42D2202A7450, VertexParallelism: 1, TaskAttemptID:attempt_1452796286596_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2EE599F5B484450DB575B247641CBE37, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F59F746641C04310AC1FDC9A779671D1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:26,789 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:26,790 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796286596_0001_1_00_000000_0
2016-01-14 18:31:26,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:26,791 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,791 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796286596_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:26,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2EE599F5B484450DB575B247641CBE37, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:26,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:26,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2EE599F5B484450DB575B247641CBE37
2016-01-14 18:31:26,792 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2EE599F5B484450DB575B247641CBE37
2016-01-14 18:31:26,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,794 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F59F746641C04310AC1FDC9A779671D1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:26,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:26,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F59F746641C04310AC1FDC9A779671D1
2016-01-14 18:31:26,795 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:26,796 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2EE599F5B484450DB575B247641CBE37
2016-01-14 18:31:26,796 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2EE599F5B484450DB575B247641CBE37
2016-01-14 18:31:26,797 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:26,797 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:26,797 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F59F746641C04310AC1FDC9A779671D1
2016-01-14 18:31:26,797 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F59F746641C04310AC1FDC9A779671D1
2016-01-14 18:31:26,801 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B762F9E0E0FA4DD9994B42D2202A7450, ordinal: 0
2016-01-14 18:31:26,801 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B762F9E0E0FA4DD9994B42D2202A7450, mem on start (mb), free: 209, total: 369, max: 672
2016-01-14 18:31:26,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:26,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:26,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:26,802 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:26,802 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:26,802 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:26,803 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:26,803 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:26,803 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:26,803 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2EE599F5B484450DB575B247641CBE37, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:26,803 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F59F746641C04310AC1FDC9A779671D1, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:26,803 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:26,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2EE599F5B484450DB575B247641CBE37 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:26,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:26,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:26,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2EE599F5B484450DB575B247641CBE37
2016-01-14 18:31:26,805 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:26,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2EE599F5B484450DB575B247641CBE37
2016-01-14 18:31:26,805 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:26,805 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796286596_0001_1_00_000000_0
2016-01-14 18:31:26,806 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:26,807 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@65e93096
2016-01-14 18:31:26,808 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@65e93096
2016-01-14 18:31:26,808 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:26,808 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:26,809 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:26,810 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2EE599F5B484450DB575B247641CBE37
2016-01-14 18:31:26,810 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 1E1E3C0084D1454394C0A8C9D6F457D8
2016-01-14 18:31:26,810 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 1E1E3C0084D1454394C0A8C9D6F457D8
2016-01-14 18:31:26,810 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@53f567d; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1472b3c; comparator=cascading.tuple.hadoop.util.TupleComparator@676770cb; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:26,811 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:26,811 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:26,811 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:26,823 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:26,823 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:26,836 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:26,836 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:26,836 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:26,837 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:26,837 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:26,837 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:26,837 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:26,838 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B762F9E0E0FA4DD9994B42D2202A7450, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:26,840 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796286596_0001_1_00_000000_0
2016-01-14 18:31:26,840 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B762F9E0E0FA4DD9994B42D2202A7450, mem on close (mb), free: 231, total: 368, max: 672
2016-01-14 18:31:26,840 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:26,840 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:26,840 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:26,840 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:26,842 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:26 Completed running task attempt: attempt_1452796286596_0001_1_00_000000_0
2016-01-14 18:31:26,844 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796286596_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:26,845 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796286596_0001_1_00, eventCount=1
2016-01-14 18:31:26,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:26,846 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B762F9E0E0FA4DD9994B42D2202A7450, taskAttemptId=attempt_1452796286596_0001_1_00_000000_0, startTime=1452796286786, finishTime=1452796286845, timeTaken=59, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286838, Process_Duration=2, Process_End_Time=1452796286840, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:26,846 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796286596_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:26,846 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:26,846 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796286596_0001_1_00_000000_0
2016-01-14 18:31:26,847 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:26,847 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:TASK_FINISHED]: vertexName=B762F9E0E0FA4DD9994B42D2202A7450, taskId=task_1452796286596_0001_1_00_000000, startTime=1452796286786, finishTime=1452796286846, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1452796286596_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286838, Process_Duration=2, Process_End_Time=1452796286840, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:26,847 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796286596_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:26,847 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450] : 1
2016-01-14 18:31:26,847 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450]
2016-01-14 18:31:26,848 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796286838, Process_Duration=2, Process_End_Time=1452796286840, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:26,848 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:26,848 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,848 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:26,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:VERTEX_FINISHED]: vertexName=B762F9E0E0FA4DD9994B42D2202A7450, vertexId=vertex_1452796286596_0001_1_00, initRequestedTime=1452796286762, initedTime=1452796286775, startRequestedTime=1452796286776, startedTime=1452796286776, finishTime=1452796286847, timeTaken=71, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286838, Process_Duration=2, Process_End_Time=1452796286840, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796286786, firstTasksToStart=[ task_1452796286596_0001_1_00_000000 ], lastTaskFinishTime=1452796286845, lastTasksToFinish=[ task_1452796286596_0001_1_00_000000 ], minTaskDuration=59, maxTaskDuration=59, avgTaskDuration=59.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796286596_0001_1_00_000000 ], longestDurationTasks=[ task_1452796286596_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:26,849 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:26,849 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:26,849 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796286596_0001_00_000001
2016-01-14 18:31:26,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:26,849 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796286596_0001_00_000001, stoppedTime=1452796286849, exitStatus=0
2016-01-14 18:31:26,850 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:26,850 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1] attempt: attempt_1452796286596_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:26,850 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796286596_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:26,850 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796286596_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:26,850 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796286596_0001_00_000001, Exiting
2016-01-14 18:31:26,850 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796286596_0001_00_000001
2016-01-14 18:31:26,850 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796286596_0001_00_000001
2016-01-14 18:31:26,850 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F59F746641C04310AC1FDC9A779671D1
2016-01-14 18:31:26,851 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1]
2016-01-14 18:31:26,851 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:26,851 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796286596_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:26,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:TASK_STARTED]: vertexName=F59F746641C04310AC1FDC9A779671D1, taskId=task_1452796286596_0001_1_01_000000, scheduledTime=1452796286851, launchTime=1452796286851
2016-01-14 18:31:26,852 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:26,852 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796286596_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:26,852 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:26,852 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1]
2016-01-14 18:31:26,853 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: F59F746641C04310AC1FDC9A779671D1, VertexParallelism: 1, TaskAttemptID:attempt_1452796286596_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B762F9E0E0FA4DD9994B42D2202A7450, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6B7650DC90D14E3BAC6318ADDC6B22C9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:26,853 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796286596_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:26,853 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:26,853 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796286596_0001_00_000002
2016-01-14 18:31:26,854 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:26,856 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796286596_0001_00_000002
2016-01-14 18:31:26,856 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796286596_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:26,856 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796286596_0001_00_000001 completed successfully
2016-01-14 18:31:26,856 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:26,858 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:26,859 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:26,859 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:26,859 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796286596_0001_00_000002, launchTime=1452796286859
2016-01-14 18:31:26,860 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796286596_0001_00_000002
2016-01-14 18:31:26,860 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796286596_0001_1_01_000000_0] to container: [container_1452796286596_0001_00_000002]
2016-01-14 18:31:26,860 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:26,861 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796286596_0001_00_000002 given task: attempt_1452796286596_0001_1_01_000000_0
2016-01-14 18:31:26,861 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796286596_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796286596_0001_1_01_000000_0
2016-01-14 18:31:26,861 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796286596_0001_1_01_000000_0] started. Is using containerId: [container_1452796286596_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:26,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F59F746641C04310AC1FDC9A779671D1, taskAttemptId=attempt_1452796286596_0001_1_01_000000_0, startTime=1452796286861, containerId=container_1452796286596_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796286596_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:26 Starting to run new task attempt: attempt_1452796286596_0001_1_01_000000_0
2016-01-14 18:31:26,861 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796286596_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:26,861 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796286596_0001_1_01_000000_0
2016-01-14 18:31:26,861 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796286596_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:26,861 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:26,862 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,862 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:26,862 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:26,862 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:26,862 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:26,862 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:26,862 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE] every/(1/1) ...yplatform/-1_ARGS_REPLACE, VertexName: F59F746641C04310AC1FDC9A779671D1, VertexParallelism: 1, TaskAttemptID:attempt_1452796286596_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B762F9E0E0FA4DD9994B42D2202A7450, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6B7650DC90D14E3BAC6318ADDC6B22C9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:26,863 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:26,863 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796286596_0001_1_01_000000_0
2016-01-14 18:31:26,863 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:26,863 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,864 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,866 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B762F9E0E0FA4DD9994B42D2202A7450, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:26,868 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:26,868 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B762F9E0E0FA4DD9994B42D2202A7450
2016-01-14 18:31:26,869 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796286596_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:26,867 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6B7650DC90D14E3BAC6318ADDC6B22C9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:26,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:26,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6B7650DC90D14E3BAC6318ADDC6B22C9
2016-01-14 18:31:26,871 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:26,871 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6B7650DC90D14E3BAC6318ADDC6B22C9
2016-01-14 18:31:26,871 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:26,871 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:26,874 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:26,874 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B762F9E0E0FA4DD9994B42D2202A7450
2016-01-14 18:31:26,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B762F9E0E0FA4DD9994B42D2202A7450
2016-01-14 18:31:26,878 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F59F746641C04310AC1FDC9A779671D1, ordinal: 1
2016-01-14 18:31:26,878 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F59F746641C04310AC1FDC9A779671D1, mem on start (mb), free: 223, total: 368, max: 672
2016-01-14 18:31:26,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:26,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:26,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:26,882 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:26,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6B7650DC90D14E3BAC6318ADDC6B22C9
2016-01-14 18:31:26,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:26,883 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:26,883 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:26,884 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:26,884 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:26,884 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:26,884 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:26,884 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6B7650DC90D14E3BAC6318ADDC6B22C9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:26,885 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B762F9E0E0FA4DD9994B42D2202A7450, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:26,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:26,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B762F9E0E0FA4DD9994B42D2202A7450 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:26,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B762F9E0E0FA4DD9994B42D2202A7450
2016-01-14 18:31:26,885 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:26,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:26,886 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:26,886 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:26,887 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B762F9E0E0FA4DD9994B42D2202A7450 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:26,887 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:26,887 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:26,887 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:26,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B762F9E0E0FA4DD9994B42D2202A7450
2016-01-14 18:31:26,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:26,889 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796286596_0001_1_01_000000_0
2016-01-14 18:31:26,890 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796286596_0001_1_00_000000_0_10105, runDuration: 48478, ]
2016-01-14 18:31:26,891 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796286596_0001_1_00_000000_0_10105] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796286891, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:26,891 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B762F9E0E0FA4DD9994B42D2202A7450
2016-01-14 18:31:26,891 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:26,892 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B762F9E0E0FA4DD9994B42D2202A7450] #18 in 2ms
2016-01-14 18:31:26,893 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:26,893 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B762F9E0E0FA4DD9994B42D2202A7450] #18, status:false, isInterrupted:false
2016-01-14 18:31:26,894 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 1E1E3C0084D1454394C0A8C9D6F457D8
2016-01-14 18:31:26,895 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE"], id: 6B7650DC90D14E3BAC6318ADDC6B22C9
2016-01-14 18:31:26,895 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:26,895 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_REPLACE"]
2016-01-14 18:31:26,895 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:26,895 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:26,896 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:26,895 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 1E1E3C0084D1454394C0A8C9D6F457D8, for 1 inputs
2016-01-14 18:31:26,896 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:26,896 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B762F9E0E0FA4DD9994B42D2202A7450
2016-01-14 18:31:26,896 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:26,896 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F59F746641C04310AC1FDC9A779671D1, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:26,897 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@18052768; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7a5e2929
2016-01-14 18:31:26,901 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:26,902 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:26,902 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796286596_0001_1_01_000000_0
2016-01-14 18:31:26,902 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796286596_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:26,903 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796286596_0001_1_01_000000_0
2016-01-14 18:31:26,903 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F59F746641C04310AC1FDC9A779671D1, mem on close (mb), free: 219, total: 368, max: 672
2016-01-14 18:31:26,904 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B762F9E0E0FA4DD9994B42D2202A7450
2016-01-14 18:31:26,904 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:26,904 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:26,904 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:26,905 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:26,905 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:26 Completed running task attempt: attempt_1452796286596_0001_1_01_000000_0
2016-01-14 18:31:26,905 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796286596_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:26,907 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796286897, Process_Duration=6, Process_End_Time=1452796286903, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:26,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F59F746641C04310AC1FDC9A779671D1, taskAttemptId=attempt_1452796286596_0001_1_01_000000_0, startTime=1452796286861, finishTime=1452796286907, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286897, Process_Duration=6, Process_End_Time=1452796286903, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:26,907 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:26,907 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796286596_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:26,908 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:26,907 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:26,908 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796286596_0001_1_01_000000_0
2016-01-14 18:31:26,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:26,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:TASK_FINISHED]: vertexName=F59F746641C04310AC1FDC9A779671D1, taskId=task_1452796286596_0001_1_01_000000, startTime=1452796286861, finishTime=1452796286908, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1452796286596_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286897, Process_Duration=6, Process_End_Time=1452796286903, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:26,909 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796286596_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:26,909 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1] : 1
2016-01-14 18:31:26,909 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1]
2016-01-14 18:31:26,910 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:26,910 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:VERTEX_FINISHED]: vertexName=F59F746641C04310AC1FDC9A779671D1, vertexId=vertex_1452796286596_0001_1_01, initRequestedTime=1452796286765, initedTime=1452796286771, startRequestedTime=1452796286779, startedTime=1452796286779, finishTime=1452796286909, timeTaken=130, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796286897, Process_Duration=6, Process_End_Time=1452796286903, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796286861, firstTasksToStart=[ task_1452796286596_0001_1_01_000000 ], lastTaskFinishTime=1452796286907, lastTasksToFinish=[ task_1452796286596_0001_1_01_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796286596_0001_1_01_000000 ], longestDurationTasks=[ task_1452796286596_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:26,910 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:26,910 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:26,910 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796286596_0001_00_000002, stoppedTime=1452796286910, exitStatus=0
2016-01-14 18:31:26,910 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:26,911 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:26,911 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796286596_0001_1
2016-01-14 18:31:26,911 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796286596_0001_1
2016-01-14 18:31:26,911 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6B7650DC90D14E3BAC6318ADDC6B22C9 for vertex: vertex_1452796286596_0001_1_01 [F59F746641C04310AC1FDC9A779671D1]
2016-01-14 18:31:26,912 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,910 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:26,913 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796286596_0001_00_000002
2016-01-14 18:31:26,913 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796286596_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:26,913 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796286596_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:26,914 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796286596_0001_00_000002, Exiting
2016-01-14 18:31:26,914 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796286596_0001_00_000002
2016-01-14 18:31:26,914 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796286596_0001_00_000002
2016-01-14 18:31:26,914 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796286596_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:26,914 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796286596_0001_00_000002 completed successfully
2016-01-14 18:31:26,918 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796286596_0001_1_00 [B762F9E0E0FA4DD9994B42D2202A7450]
2016-01-14 18:31:26,920 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796286596_0001_1, queueSize=0
2016-01-14 18:31:26,921 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796286596_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796286596_0001_1, startTime=1452796286762, finishTime=1452796286918, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=520, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=11, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=772800512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592573735, Process_Duration=8, Process_End_Time=2905592573743, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=3, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:26,921 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796286596_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:26,921 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:26,921 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796286596_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:26,921 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796286596_0001_00_000002
2016-01-14 18:31:26,922 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796286596_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:26 Completed Dag: dag_1452796286596_0001_1
2016-01-14 18:31:26,922 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796286596_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:26,923 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796286596_0001_1_post
2016-01-14 18:31:26,923 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:26,923 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:26,929 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/-1_ARGS_REPLACE, applicationId=application_1452796286596_0001
2016-01-14 18:31:26,931 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:26,931 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:26,931 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:26,931 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:26,931 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:26,932 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:26,932 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:26,932 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:26,932 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:26,932 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:26,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:26,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:26,935 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:26,937 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D8C8FFB0F1704DBCA8FDFB5E97917235/CCE0A1F1C142450D837F355E69C367BE/.tez/application_1452796286596_0001
2016-01-14 18:31:26,938 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.362
2016-01-14 18:31:27,027 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_REPLACE
2016-01-14 18:31:27,066 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:27,066 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:27,108 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 18:31:27,122 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-01-14 18:31:27,122 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:27,123 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:27,123 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:27,140 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:27,146 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP
2016-01-14 18:31:27,200 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:27,200 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:27,200 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP"]
2016-01-14 18:31:27,200 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:27,200 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:27,200 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:27,201 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...mblyplatform/-1_ARGS_SWAP
2016-01-14 18:31:27,201 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:27,215 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:27,220 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:27,220 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:27,220 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:27,230 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C/.tez/application_1452796287220_0001 doesn't exist and is created
2016-01-14 18:31:27,243 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:27,244 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:27,244 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C/.tez/application_1452796287220_0001
2016-01-14 18:31:27,245 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796287220_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:27,256 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:27,257 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:27,258 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:27,258 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:27,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C/.tez/application_1452796287220_0001/localmode-log-dir
2016-01-14 18:31:27,259 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C/.tez/application_1452796287220_0001/localmode-log-dir/history.txt.appattempt_1452796287220_0001_000000, maxErrors=10
2016-01-14 18:31:27,259 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:27,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796287220_0001_000000, appSubmitTime=1452796287245, launchTime=1452796287245
2016-01-14 18:31:27,263 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:27,263 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:27,264 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:27,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:27,265 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:38265
2016-01-14 18:31:27,267 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:27,269 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796287220_0001_000000, startTime=1452796287269
2016-01-14 18:31:27,269 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:27,269 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:27,344 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:27,344 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:27,344 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1452796287220_0001, dagName=[C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-01-14 18:31:27,360 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-01-14 18:31:27,362 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796287220_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C/.tez/application_1452796287220_0001/localmode-log-dir/dag_1452796287220_0001_1.dot
2016-01-14 18:31:27,362 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C/.tez/application_1452796287220_0001/localmode-log-dir/dag_1452796287220_0001_1-tez-dag.pb.txt
2016-01-14 18:31:27,368 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796287220_0001_1
2016-01-14 18:31:27,369 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:27,369 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-01-14 18:31:27 Running Dag: dag_1452796287220_0001_1
2016-01-14 18:31:27,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796287220_0001_1, submitTime=1452796287360
2016-01-14 18:31:27,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:27,379 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C9C63592B1FB40E4883077D5BDD1DA3B
2016-01-14 18:31:27,380 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:27,380 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:27,381 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796287220_0001_1, initTime=1452796287379
2016-01-14 18:31:27,381 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796287220_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:27,381 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:DAG_STARTED]: dagID=dag_1452796287220_0001_1, startTime=1452796287381
2016-01-14 18:31:27,382 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:27,381 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:27,382 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796287220_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:27,383 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C8D68DB5432F44BE8A11F0C7497ED298 : {F0F525BDEB7547C6BD07C1BCEEF2EADE={InputName=F0F525BDEB7547C6BD07C1BCEEF2EADE}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:27,384 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F0F525BDEB7547C6BD07C1BCEEF2EADE, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:27,384 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298]
2016-01-14 18:31:27,384 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298]
2016-01-14 18:31:27,384 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298]
2016-01-14 18:31:27,384 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1452796287220_0001, dagName=[C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C] every/(1/1) ...mblyplatform/-1_ARGS_SWAP
2016-01-14 18:31:27,384 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298]. Starting root input initializers: 1
2016-01-14 18:31:27,386 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:27,386 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B]
2016-01-14 18:31:27,387 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F0F525BDEB7547C6BD07C1BCEEF2EADE on vertex vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298]
2016-01-14 18:31:27,388 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:27,389 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:27,389 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B]
2016-01-14 18:31:27,389 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B]
2016-01-14 18:31:27,389 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B]
2016-01-14 18:31:27,389 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A155A596A7794E58A4E813A511B727CB, vertexId=vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:27,389 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F0F525BDEB7547C6BD07C1BCEEF2EADE asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:27,389 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A155A596A7794E58A4E813A511B727CB, vertexId=vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B]
2016-01-14 18:31:27,390 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:27,390 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:27,405 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:27,406 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F0F525BDEB7547C6BD07C1BCEEF2EADE on vertex vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298]
2016-01-14 18:31:27,406 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A155A596A7794E58A4E813A511B727CB, vertexName=C9C63592B1FB40E4883077D5BDD1DA3B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:27,407 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A155A596A7794E58A4E813A511B727CB, vertexId=vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B]
2016-01-14 18:31:27,408 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:27,408 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C9C63592B1FB40E4883077D5BDD1DA3B, vertexId=vertex_1452796287220_0001_1_01, initRequestedTime=1452796287386, initedTime=1452796287408, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:27,408 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:27,409 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:27,409 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F0F525BDEB7547C6BD07C1BCEEF2EADE=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:27,409 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298] parallelism set to 1
2016-01-14 18:31:27,409 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:27,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796287220_0001_1_00, eventCount=1
2016-01-14 18:31:27,410 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C8D68DB5432F44BE8A11F0C7497ED298, vertexId=vertex_1452796287220_0001_1_00, initRequestedTime=1452796287383, initedTime=1452796287410, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:27,410 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298] with distanceFromRoot: 0
2016-01-14 18:31:27,410 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:27,411 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:27,411 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C8D68DB5432F44BE8A11F0C7497ED298
2016-01-14 18:31:27,411 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298]
2016-01-14 18:31:27,412 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796287220_0001_1_00, startRequestedTime=1452796287411, startedTime=1452796287411
2016-01-14 18:31:27,412 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:27,412 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:27,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:27,413 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:TASK_STARTED]: vertexName=C8D68DB5432F44BE8A11F0C7497ED298, taskId=task_1452796287220_0001_1_00_000000, scheduledTime=1452796287413, launchTime=1452796287413
2016-01-14 18:31:27,413 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287220_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:27,414 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796287220_0001_1_00 for vertex: vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B] numStartedSources: 1 numSources: 1
2016-01-14 18:31:27,414 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B] with distanceFromRoot: 1
2016-01-14 18:31:27,414 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298]
2016-01-14 18:31:27,415 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C8D68DB5432F44BE8A11F0C7497ED298 in vertex: C9C63592B1FB40E4883077D5BDD1DA3B
2016-01-14 18:31:27,415 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C9C63592B1FB40E4883077D5BDD1DA3B with 1 source tasks and 1 pending tasks
2016-01-14 18:31:27,415 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:27,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796287220_0001_1_01, startRequestedTime=1452796287414, startedTime=1452796287414
2016-01-14 18:31:27,416 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:27,416 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: C8D68DB5432F44BE8A11F0C7497ED298, VertexParallelism: 1, TaskAttemptID:attempt_1452796287220_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F0F525BDEB7547C6BD07C1BCEEF2EADE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C9C63592B1FB40E4883077D5BDD1DA3B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:27,416 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287220_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:27,417 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:27,417 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:27,418 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:27,418 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:27,418 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796287220_0001_00_000001
2016-01-14 18:31:27,419 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:27,420 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796287220_0001_00_000001
2016-01-14 18:31:27,421 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:27,421 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796287220_0001, with dag id: dag_1452796287220_0001_1
2016-01-14 18:31:27,422 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:27,422 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:27,422 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796287220_0001_00_000001, launchTime=1452796287422
2016-01-14 18:31:27,423 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796287220_0001_00_000001
2016-01-14 18:31:27,424 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796287220_0001_1_00_000000_0] to container: [container_1452796287220_0001_00_000001]
2016-01-14 18:31:27,424 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:27,424 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796287220_0001_00_000001 given task: attempt_1452796287220_0001_1_00_000000_0
2016-01-14 18:31:27,425 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796287220_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796287220_0001_1_00_000000_0
2016-01-14 18:31:27 Starting to run new task attempt: attempt_1452796287220_0001_1_00_000000_0
2016-01-14 18:31:27,426 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796287220_0001_1_00_000000_0
2016-01-14 18:31:27,425 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796287220_0001_1_00_000000_0] started. Is using containerId: [container_1452796287220_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:27,426 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:27,426 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C8D68DB5432F44BE8A11F0C7497ED298, taskAttemptId=attempt_1452796287220_0001_1_00_000000_0, startTime=1452796287425, containerId=container_1452796287220_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796287220_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:27,426 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:27,426 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287220_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:27,426 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:27,426 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287220_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:27,426 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:27,427 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:27,427 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:27,427 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:27,427 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: C8D68DB5432F44BE8A11F0C7497ED298, VertexParallelism: 1, TaskAttemptID:attempt_1452796287220_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F0F525BDEB7547C6BD07C1BCEEF2EADE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C9C63592B1FB40E4883077D5BDD1DA3B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:27,427 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:27,428 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796287220_0001_1_00_000000_0
2016-01-14 18:31:27,428 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:27,429 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:27,429 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796287220_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:27,430 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F0F525BDEB7547C6BD07C1BCEEF2EADE, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:27,431 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:27,431 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F0F525BDEB7547C6BD07C1BCEEF2EADE
2016-01-14 18:31:27,431 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F0F525BDEB7547C6BD07C1BCEEF2EADE
2016-01-14 18:31:27,431 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C9C63592B1FB40E4883077D5BDD1DA3B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:27,430 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:27,434 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:27,434 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F0F525BDEB7547C6BD07C1BCEEF2EADE
2016-01-14 18:31:27,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F0F525BDEB7547C6BD07C1BCEEF2EADE
2016-01-14 18:31:27,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:27,435 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C9C63592B1FB40E4883077D5BDD1DA3B
2016-01-14 18:31:27,436 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:27,437 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:27,437 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C9C63592B1FB40E4883077D5BDD1DA3B
2016-01-14 18:31:27,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C9C63592B1FB40E4883077D5BDD1DA3B
2016-01-14 18:31:27,442 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C8D68DB5432F44BE8A11F0C7497ED298, ordinal: 0
2016-01-14 18:31:27,442 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8D68DB5432F44BE8A11F0C7497ED298, mem on start (mb), free: 323, total: 368, max: 672
2016-01-14 18:31:27,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:27,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:27,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:27,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:27,444 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:27,444 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:27,444 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:27,445 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:27,445 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:27,446 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:27,446 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F0F525BDEB7547C6BD07C1BCEEF2EADE, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:27,446 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C9C63592B1FB40E4883077D5BDD1DA3B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:27,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:27,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F0F525BDEB7547C6BD07C1BCEEF2EADE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:27,447 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F0F525BDEB7547C6BD07C1BCEEF2EADE
2016-01-14 18:31:27,447 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:27,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F0F525BDEB7547C6BD07C1BCEEF2EADE
2016-01-14 18:31:27,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:27,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:27,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:27,450 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796287220_0001_1_00_000000_0
2016-01-14 18:31:27,450 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:27,450 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@49764d6
2016-01-14 18:31:27,451 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@49764d6
2016-01-14 18:31:27,451 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:27,451 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:27,454 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:27,456 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F0F525BDEB7547C6BD07C1BCEEF2EADE
2016-01-14 18:31:27,456 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 5F5F18B20A1A4288A4C5ED3A769ACAB9
2016-01-14 18:31:27,456 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 5F5F18B20A1A4288A4C5ED3A769ACAB9
2016-01-14 18:31:27,457 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7f127e0; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6970e732; comparator=cascading.tuple.hadoop.util.TupleComparator@735ab9b2; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:27,457 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:27,457 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:27,458 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:27,471 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:27,472 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:27,472 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:27,472 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:27,472 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:27,473 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:27,473 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:27,474 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C8D68DB5432F44BE8A11F0C7497ED298, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:27,476 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796287220_0001_1_00_000000_0
2016-01-14 18:31:27,476 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8D68DB5432F44BE8A11F0C7497ED298, mem on close (mb), free: 221, total: 368, max: 672
2016-01-14 18:31:27,477 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:27,477 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:27,477 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:27,477 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:27,479 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:27 Completed running task attempt: attempt_1452796287220_0001_1_00_000000_0
2016-01-14 18:31:27,484 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796287220_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:27,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796287220_0001_1_00, eventCount=1
2016-01-14 18:31:27,485 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796287474, Process_Duration=2, Process_End_Time=1452796287476, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:27,485 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:27,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C8D68DB5432F44BE8A11F0C7497ED298, taskAttemptId=attempt_1452796287220_0001_1_00_000000_0, startTime=1452796287425, finishTime=1452796287485, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796287474, Process_Duration=2, Process_End_Time=1452796287476, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:27,486 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287220_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:27,486 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796287220_0001_1_00_000000_0
2016-01-14 18:31:27,486 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:27,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:TASK_FINISHED]: vertexName=C8D68DB5432F44BE8A11F0C7497ED298, taskId=task_1452796287220_0001_1_00_000000, startTime=1452796287425, finishTime=1452796287486, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1452796287220_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796287474, Process_Duration=2, Process_End_Time=1452796287476, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:27,487 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287220_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:27,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:27,487 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:27,487 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:27,487 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796287220_0001_00_000001
2016-01-14 18:31:27,487 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298] : 1
2016-01-14 18:31:27,488 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298]
2016-01-14 18:31:27,489 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:27,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:VERTEX_FINISHED]: vertexName=C8D68DB5432F44BE8A11F0C7497ED298, vertexId=vertex_1452796287220_0001_1_00, initRequestedTime=1452796287383, initedTime=1452796287410, startRequestedTime=1452796287411, startedTime=1452796287411, finishTime=1452796287489, timeTaken=78, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796287474, Process_Duration=2, Process_End_Time=1452796287476, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796287425, firstTasksToStart=[ task_1452796287220_0001_1_00_000000 ], lastTaskFinishTime=1452796287485, lastTasksToFinish=[ task_1452796287220_0001_1_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796287220_0001_1_00_000000 ], longestDurationTasks=[ task_1452796287220_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:27,490 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:27,490 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:27,490 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B] attempt: attempt_1452796287220_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:27,491 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C9C63592B1FB40E4883077D5BDD1DA3B
2016-01-14 18:31:27,491 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B]
2016-01-14 18:31:27,491 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796287220_0001_00_000001, stoppedTime=1452796287491, exitStatus=0
2016-01-14 18:31:27,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:27,491 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:27,492 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:27,492 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:TASK_STARTED]: vertexName=C9C63592B1FB40E4883077D5BDD1DA3B, taskId=task_1452796287220_0001_1_01_000000, scheduledTime=1452796287492, launchTime=1452796287492
2016-01-14 18:31:27,492 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287220_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:27,492 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:27,493 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796287220_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:27,493 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B]
2016-01-14 18:31:27,493 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:27,493 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: C9C63592B1FB40E4883077D5BDD1DA3B, VertexParallelism: 1, TaskAttemptID:attempt_1452796287220_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8D68DB5432F44BE8A11F0C7497ED298, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A155A596A7794E58A4E813A511B727CB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:27,493 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287220_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:27,493 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:27,494 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796287220_0001_00_000002
2016-01-14 18:31:27,494 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:27,496 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796287220_0001_00_000002
2016-01-14 18:31:27,496 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:27,496 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:27,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:27,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796287220_0001_00_000002, launchTime=1452796287496
2016-01-14 18:31:27,497 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796287220_0001_00_000002
2016-01-14 18:31:27,497 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796287220_0001_1_01_000000_0] to container: [container_1452796287220_0001_00_000002]
2016-01-14 18:31:27,498 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796287220_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:27,498 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796287220_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:27,498 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796287220_0001_00_000001, Exiting
2016-01-14 18:31:27,498 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796287220_0001_00_000001
2016-01-14 18:31:27,498 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796287220_0001_00_000001
2016-01-14 18:31:27,498 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:27,499 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796287220_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:27,499 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796287220_0001_00_000002 given task: attempt_1452796287220_0001_1_01_000000_0
2016-01-14 18:31:27,499 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796287220_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796287220_0001_1_01_000000_0
2016-01-14 18:31:27,499 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796287220_0001_00_000001 completed successfully
2016-01-14 18:31:27,499 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796287220_0001_1_01_000000_0] started. Is using containerId: [container_1452796287220_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:27 Starting to run new task attempt: attempt_1452796287220_0001_1_01_000000_0
2016-01-14 18:31:27,500 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796287220_0001_1_01_000000_0
2016-01-14 18:31:27,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C9C63592B1FB40E4883077D5BDD1DA3B, taskAttemptId=attempt_1452796287220_0001_1_01_000000_0, startTime=1452796287499, containerId=container_1452796287220_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796287220_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:27,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:27,500 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287220_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:27,500 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:27,500 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287220_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:27,500 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:27,500 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:27,500 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:27,501 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:27,501 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:27,501 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:27,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C] every/(1/1) ...mblyplatform/-1_ARGS_SWAP, VertexName: C9C63592B1FB40E4883077D5BDD1DA3B, VertexParallelism: 1, TaskAttemptID:attempt_1452796287220_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8D68DB5432F44BE8A11F0C7497ED298, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A155A596A7794E58A4E813A511B727CB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:27,501 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:27,502 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796287220_0001_1_01_000000_0
2016-01-14 18:31:27,502 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:27,502 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:27,502 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796287220_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:27,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C8D68DB5432F44BE8A11F0C7497ED298, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:27,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:27,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C8D68DB5432F44BE8A11F0C7497ED298
2016-01-14 18:31:27,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:27,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A155A596A7794E58A4E813A511B727CB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:27,505 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:27,505 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A155A596A7794E58A4E813A511B727CB
2016-01-14 18:31:27,505 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:27,505 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A155A596A7794E58A4E813A511B727CB
2016-01-14 18:31:27,508 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:27,515 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:27,515 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A155A596A7794E58A4E813A511B727CB
2016-01-14 18:31:27,515 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:27,516 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:27,516 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C8D68DB5432F44BE8A11F0C7497ED298
2016-01-14 18:31:27,517 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C8D68DB5432F44BE8A11F0C7497ED298
2016-01-14 18:31:27,517 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C9C63592B1FB40E4883077D5BDD1DA3B, ordinal: 1
2016-01-14 18:31:27,517 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C9C63592B1FB40E4883077D5BDD1DA3B, mem on start (mb), free: 216, total: 368, max: 672
2016-01-14 18:31:27,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:27,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:27,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:27,518 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:27,518 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:27,519 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:27,519 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:27,519 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:27,519 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:27,520 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:27,520 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A155A596A7794E58A4E813A511B727CB, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:27,520 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C8D68DB5432F44BE8A11F0C7497ED298, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:27,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:27,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C8D68DB5432F44BE8A11F0C7497ED298
2016-01-14 18:31:27,521 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C8D68DB5432F44BE8A11F0C7497ED298 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:27,522 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:27,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:27,526 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:27,526 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:27,527 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C8D68DB5432F44BE8A11F0C7497ED298 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:27,527 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:27,527 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:27,527 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:27,529 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C8D68DB5432F44BE8A11F0C7497ED298
2016-01-14 18:31:27,529 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:27,531 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796287220_0001_1_01_000000_0
2016-01-14 18:31:27,532 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796287220_0001_1_00_000000_0_10111, runDuration: 46211, ]
2016-01-14 18:31:27,535 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:27,536 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 5F5F18B20A1A4288A4C5ED3A769ACAB9
2016-01-14 18:31:27,536 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP"], id: A155A596A7794E58A4E813A511B727CB
2016-01-14 18:31:27,536 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/-1_ARGS_SWAP"]
2016-01-14 18:31:27,536 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 5F5F18B20A1A4288A4C5ED3A769ACAB9, for 1 inputs
2016-01-14 18:31:27,538 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796287220_0001_1_00_000000_0_10111] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796287538, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:27,538 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C8D68DB5432F44BE8A11F0C7497ED298
2016-01-14 18:31:27,538 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:27,538 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C8D68DB5432F44BE8A11F0C7497ED298] #19 in 1ms
2016-01-14 18:31:27,539 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C8D68DB5432F44BE8A11F0C7497ED298] #19, status:false, isInterrupted:false
2016-01-14 18:31:27,540 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:27,540 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:27,540 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:27,540 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:27,541 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:27,541 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C8D68DB5432F44BE8A11F0C7497ED298
2016-01-14 18:31:27,541 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:27,541 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C9C63592B1FB40E4883077D5BDD1DA3B, all 1 inputs ready in: 00:00:00.005
2016-01-14 18:31:27,542 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@37857368; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1a5bcd52
2016-01-14 18:31:27,546 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:27,547 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:27,547 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796287220_0001_1_01_000000_0
2016-01-14 18:31:27,548 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796287220_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:27,548 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796287220_0001_1_01_000000_0
2016-01-14 18:31:27,549 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C9C63592B1FB40E4883077D5BDD1DA3B, mem on close (mb), free: 213, total: 368, max: 672
2016-01-14 18:31:27,549 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C8D68DB5432F44BE8A11F0C7497ED298
2016-01-14 18:31:27,549 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:27,550 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:27,550 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:27,550 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:27,551 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:27 Completed running task attempt: attempt_1452796287220_0001_1_01_000000_0
2016-01-14 18:31:27,552 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796287220_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:27,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796287541, Process_Duration=7, Process_End_Time=1452796287548, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:27,553 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C9C63592B1FB40E4883077D5BDD1DA3B, taskAttemptId=attempt_1452796287220_0001_1_01_000000_0, startTime=1452796287499, finishTime=1452796287553, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796287541, Process_Duration=7, Process_End_Time=1452796287548, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:27,554 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287220_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:27,554 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796287220_0001_1_01_000000_0
2016-01-14 18:31:27,554 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:27,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:TASK_FINISHED]: vertexName=C9C63592B1FB40E4883077D5BDD1DA3B, taskId=task_1452796287220_0001_1_01_000000, startTime=1452796287499, finishTime=1452796287554, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1452796287220_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796287541, Process_Duration=7, Process_End_Time=1452796287548, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:27,555 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287220_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:27,555 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B] : 1
2016-01-14 18:31:27,555 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B]
2016-01-14 18:31:27,556 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:VERTEX_FINISHED]: vertexName=C9C63592B1FB40E4883077D5BDD1DA3B, vertexId=vertex_1452796287220_0001_1_01, initRequestedTime=1452796287386, initedTime=1452796287408, startRequestedTime=1452796287414, startedTime=1452796287414, finishTime=1452796287555, timeTaken=141, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=187, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796287541, Process_Duration=7, Process_End_Time=1452796287548, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796287499, firstTasksToStart=[ task_1452796287220_0001_1_01_000000 ], lastTaskFinishTime=1452796287553, lastTasksToFinish=[ task_1452796287220_0001_1_01_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796287220_0001_1_01_000000 ], longestDurationTasks=[ task_1452796287220_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:27,556 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:27,557 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:27,553 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:27,557 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:27,558 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:27,559 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796287220_0001_00_000002, stoppedTime=1452796287559, exitStatus=0
2016-01-14 18:31:27,559 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:27,560 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:27,560 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796287220_0001_1
2016-01-14 18:31:27,561 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796287220_0001_1
2016-01-14 18:31:27,561 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: A155A596A7794E58A4E813A511B727CB for vertex: vertex_1452796287220_0001_1_01 [C9C63592B1FB40E4883077D5BDD1DA3B]
2016-01-14 18:31:27,559 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:27,563 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:27,564 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:27,564 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796287220_0001_00_000002
2016-01-14 18:31:27,565 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796287220_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:27,565 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796287220_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:27,565 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796287220_0001_00_000002, Exiting
2016-01-14 18:31:27,565 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796287220_0001_00_000002
2016-01-14 18:31:27,565 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796287220_0001_00_000002
2016-01-14 18:31:27,565 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796287220_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:27,566 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796287220_0001_00_000002 completed successfully
2016-01-14 18:31:27,570 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796287220_0001_1_00 [C8D68DB5432F44BE8A11F0C7497ED298]
2016-01-14 18:31:27,572 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796287220_0001_1, queueSize=0
2016-01-14 18:31:27,573 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:27,573 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287220_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796287220_0001_1, startTime=1452796287381, finishTime=1452796287570, timeTaken=189, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1040, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=772800512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=12, MERGE_PHASE_TIME=14, FIRST_EVENT_RECEIVED=10, LAST_EVENT_RECEIVED=10, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592575015, Process_Duration=9, Process_End_Time=2905592575024, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:27,573 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796287220_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:27,573 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796287220_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:27,574 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796287220_0001_00_000002
2016-01-14 18:31:27,574 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287220_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:27 Completed Dag: dag_1452796287220_0001_1
2016-01-14 18:31:27,575 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796287220_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:27,575 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796287220_0001_1_post
2016-01-14 18:31:27,575 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:27,575 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:27,580 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/-1_ARGS_SWAP, applicationId=application_1452796287220_0001
2016-01-14 18:31:27,582 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:27,582 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:27,582 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:27,582 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:27,582 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:27,583 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:27,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:27,583 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:27,583 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:27,584 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:27,584 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:27,584 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:27,584 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:27,589 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C11E77710D7746889FAA7964812128B2/A0080A8366934DD3A9DDB5B77362519C/.tez/application_1452796287220_0001
2016-01-14 18:31:27,589 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.390
2016-01-14 18:31:27,660 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//-1_ARGS_SWAP
2016-01-14 18:31:27,682 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:27,683 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:27,684 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:27,684 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:27,722 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 18:31:27,732 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-01-14 18:31:27,732 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:27,733 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:27,733 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:27,755 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:27,764 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1
2016-01-14 18:31:27,837 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:27,837 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:27,838 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1"]
2016-01-14 18:31:27,838 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:27,838 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:27,838 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:27,840 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...mblyplatform/0_'field'_-1
2016-01-14 18:31:27,840 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:27,854 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:27,865 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:27,865 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:27,865 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:27,875 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB/.tez/application_1452796287865_0001 doesn't exist and is created
2016-01-14 18:31:27,889 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:27,889 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:27,889 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB/.tez/application_1452796287865_0001
2016-01-14 18:31:27,890 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796287865_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:27,901 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:27,902 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:27,903 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:27,903 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:27,903 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB/.tez/application_1452796287865_0001/localmode-log-dir
2016-01-14 18:31:27,903 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB/.tez/application_1452796287865_0001/localmode-log-dir/history.txt.appattempt_1452796287865_0001_000000, maxErrors=10
2016-01-14 18:31:27,904 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:27,904 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796287865_0001_000000, appSubmitTime=1452796287890, launchTime=1452796287890
2016-01-14 18:31:27,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:27,909 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:38490
2016-01-14 18:31:27,911 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:27,913 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:27,913 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:27,914 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:27,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796287865_0001_000000, startTime=1452796287917
2016-01-14 18:31:27,917 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:27,917 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:27,989 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:27,989 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:27,989 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1452796287865_0001, dagName=[662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB] every/(1/1) ...mblyplatform/0_'field'_-1
2016-01-14 18:31:28,005 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB] every/(1/1) ...mblyplatform/0_'field'_-1
2016-01-14 18:31:28,006 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796287865_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB/.tez/application_1452796287865_0001/localmode-log-dir/dag_1452796287865_0001_1.dot
2016-01-14 18:31:28,007 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB/.tez/application_1452796287865_0001/localmode-log-dir/dag_1452796287865_0001_1-tez-dag.pb.txt
2016-01-14 18:31:28,023 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796287865_0001_1
2016-01-14 18:31:28,023 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,023 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB] every/(1/1) ...mblyplatform/0_'field'_-1
2016-01-14 18:31:28 Running Dag: dag_1452796287865_0001_1
2016-01-14 18:31:28,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796287865_0001_1, submitTime=1452796288005
2016-01-14 18:31:28,032 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:28,033 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E427BD1C382A4B65824E761E214915DA
2016-01-14 18:31:28,033 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:28,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796287865_0001_1, initTime=1452796288033
2016-01-14 18:31:28,034 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796287865_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:28,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:28,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:DAG_STARTED]: dagID=dag_1452796287865_0001_1, startTime=1452796288034
2016-01-14 18:31:28,035 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:28,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:28,035 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796287865_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:28,035 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: EE8E299B3EA54ED4BD22969EB6C0C20B : {CF72663ACBC7417EB7A1DA92000AE279={InputName=CF72663ACBC7417EB7A1DA92000AE279}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:28,035 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: CF72663ACBC7417EB7A1DA92000AE279, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:28,035 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B]
2016-01-14 18:31:28,035 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B]
2016-01-14 18:31:28,035 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B]
2016-01-14 18:31:28,036 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B]. Starting root input initializers: 1
2016-01-14 18:31:28,036 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:28,037 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA]
2016-01-14 18:31:28,037 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: CF72663ACBC7417EB7A1DA92000AE279 on vertex vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B]
2016-01-14 18:31:28,037 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1452796287865_0001, dagName=[662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB] every/(1/1) ...mblyplatform/0_'field'_-1
2016-01-14 18:31:28,038 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:28,039 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input CF72663ACBC7417EB7A1DA92000AE279 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:28,039 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:28,039 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:28,041 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:28,041 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: CF72663ACBC7417EB7A1DA92000AE279 on vertex vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B]
2016-01-14 18:31:28,042 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:28,042 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA]
2016-01-14 18:31:28,042 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA]
2016-01-14 18:31:28,042 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA]
2016-01-14 18:31:28,042 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=BD592F6D3A2A4610828933711FC6035B, vertexId=vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:28,042 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=BD592F6D3A2A4610828933711FC6035B, vertexId=vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA]
2016-01-14 18:31:28,045 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=BD592F6D3A2A4610828933711FC6035B, vertexName=E427BD1C382A4B65824E761E214915DA, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:28,045 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=BD592F6D3A2A4610828933711FC6035B, vertexId=vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA]
2016-01-14 18:31:28,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:28,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E427BD1C382A4B65824E761E214915DA, vertexId=vertex_1452796287865_0001_1_01, initRequestedTime=1452796288037, initedTime=1452796288046, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:28,046 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:28,046 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:28,047 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {CF72663ACBC7417EB7A1DA92000AE279=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:28,047 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B] parallelism set to 1
2016-01-14 18:31:28,047 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:28,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796287865_0001_1_00, eventCount=1
2016-01-14 18:31:28,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:28,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EE8E299B3EA54ED4BD22969EB6C0C20B, vertexId=vertex_1452796287865_0001_1_00, initRequestedTime=1452796288035, initedTime=1452796288048, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:28,048 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B] with distanceFromRoot: 0
2016-01-14 18:31:28,048 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:28,049 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in EE8E299B3EA54ED4BD22969EB6C0C20B
2016-01-14 18:31:28,049 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B]
2016-01-14 18:31:28,049 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:28,049 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796287865_0001_1_00, startRequestedTime=1452796288049, startedTime=1452796288049
2016-01-14 18:31:28,050 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:28,050 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:28,050 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:TASK_STARTED]: vertexName=EE8E299B3EA54ED4BD22969EB6C0C20B, taskId=task_1452796287865_0001_1_00_000000, scheduledTime=1452796288050, launchTime=1452796288050
2016-01-14 18:31:28,051 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287865_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:28,051 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796287865_0001_1_00 for vertex: vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA] numStartedSources: 1 numSources: 1
2016-01-14 18:31:28,051 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA] with distanceFromRoot: 1
2016-01-14 18:31:28,051 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B]
2016-01-14 18:31:28,054 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: EE8E299B3EA54ED4BD22969EB6C0C20B in vertex: E427BD1C382A4B65824E761E214915DA
2016-01-14 18:31:28,054 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E427BD1C382A4B65824E761E214915DA with 1 source tasks and 1 pending tasks
2016-01-14 18:31:28,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:28,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796287865_0001_1_01, startRequestedTime=1452796288051, startedTime=1452796288051
2016-01-14 18:31:28,055 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:28,055 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: EE8E299B3EA54ED4BD22969EB6C0C20B, VertexParallelism: 1, TaskAttemptID:attempt_1452796287865_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CF72663ACBC7417EB7A1DA92000AE279, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E427BD1C382A4B65824E761E214915DA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:28,055 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287865_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:28,055 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:28,056 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:28,056 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:28,056 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:28,057 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796287865_0001_00_000001
2016-01-14 18:31:28,057 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:28,059 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796287865_0001_00_000001
2016-01-14 18:31:28,059 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:28,059 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:28,060 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:28,060 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796287865_0001_00_000001, launchTime=1452796288059
2016-01-14 18:31:28,060 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796287865_0001_00_000001
2016-01-14 18:31:28,061 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796287865_0001_1_00_000000_0] to container: [container_1452796287865_0001_00_000001]
2016-01-14 18:31:28,061 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:28,061 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796287865_0001_00_000001 given task: attempt_1452796287865_0001_1_00_000000_0
2016-01-14 18:31:28,061 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796287865_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796287865_0001_1_00_000000_0
2016-01-14 18:31:28,062 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796287865_0001_1_00_000000_0] started. Is using containerId: [container_1452796287865_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:28,062 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:28,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EE8E299B3EA54ED4BD22969EB6C0C20B, taskAttemptId=attempt_1452796287865_0001_1_00_000000_0, startTime=1452796288062, containerId=container_1452796287865_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796287865_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:28 Starting to run new task attempt: attempt_1452796287865_0001_1_00_000000_0
2016-01-14 18:31:28,063 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796287865_0001_1_00_000000_0
2016-01-14 18:31:28,063 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287865_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:28,063 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,063 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287865_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:28,063 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:28,065 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:28,065 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:28,066 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:28,066 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:28,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: EE8E299B3EA54ED4BD22969EB6C0C20B, VertexParallelism: 1, TaskAttemptID:attempt_1452796287865_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CF72663ACBC7417EB7A1DA92000AE279, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E427BD1C382A4B65824E761E214915DA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:28,066 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:28,066 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796287865_0001_1_00_000000_0
2016-01-14 18:31:28,066 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:28,067 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,068 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CF72663ACBC7417EB7A1DA92000AE279, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:28,069 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796287865_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:28,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:28,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E427BD1C382A4B65824E761E214915DA, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:28,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:28,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E427BD1C382A4B65824E761E214915DA
2016-01-14 18:31:28,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CF72663ACBC7417EB7A1DA92000AE279
2016-01-14 18:31:28,071 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: CF72663ACBC7417EB7A1DA92000AE279
2016-01-14 18:31:28,071 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:28,072 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E427BD1C382A4B65824E761E214915DA
2016-01-14 18:31:28,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E427BD1C382A4B65824E761E214915DA
2016-01-14 18:31:28,073 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:28,074 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796287865_0001, with dag id: dag_1452796287865_0001_1
2016-01-14 18:31:28,075 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:28,075 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: CF72663ACBC7417EB7A1DA92000AE279
2016-01-14 18:31:28,075 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CF72663ACBC7417EB7A1DA92000AE279
2016-01-14 18:31:28,082 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EE8E299B3EA54ED4BD22969EB6C0C20B, ordinal: 0
2016-01-14 18:31:28,082 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE8E299B3EA54ED4BD22969EB6C0C20B, mem on start (mb), free: 317, total: 369, max: 672
2016-01-14 18:31:28,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:28,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:28,082 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:28,082 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:28,083 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:28,083 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:28,083 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:28,083 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:28,083 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:28,083 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CF72663ACBC7417EB7A1DA92000AE279, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:28,083 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E427BD1C382A4B65824E761E214915DA, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:28,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:28,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CF72663ACBC7417EB7A1DA92000AE279 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:28,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:28,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:28,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CF72663ACBC7417EB7A1DA92000AE279
2016-01-14 18:31:28,084 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:28,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CF72663ACBC7417EB7A1DA92000AE279
2016-01-14 18:31:28,084 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:28,085 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796287865_0001_1_00_000000_0
2016-01-14 18:31:28,085 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:28,085 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@62c63720
2016-01-14 18:31:28,086 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@62c63720
2016-01-14 18:31:28,087 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:28,087 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:28,088 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:28,089 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: CF72663ACBC7417EB7A1DA92000AE279
2016-01-14 18:31:28,089 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C941BF295D4447B1B25964CF92D5ECB1
2016-01-14 18:31:28,089 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C941BF295D4447B1B25964CF92D5ECB1
2016-01-14 18:31:28,090 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1478232f; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5b4ded28; comparator=cascading.tuple.hadoop.util.TupleComparator@590d0bec; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:28,090 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:28,090 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:28,090 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:28,104 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:28,104 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:28,105 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:28,105 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:28,105 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:28,106 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:28,107 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:28,107 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EE8E299B3EA54ED4BD22969EB6C0C20B, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:28,110 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796287865_0001_1_00_000000_0
2016-01-14 18:31:28,110 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EE8E299B3EA54ED4BD22969EB6C0C20B, mem on close (mb), free: 215, total: 369, max: 672
2016-01-14 18:31:28,110 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:28,110 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:28,110 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:28,111 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:28,112 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:28 Completed running task attempt: attempt_1452796287865_0001_1_00_000000_0
2016-01-14 18:31:28,115 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796287865_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:28,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796287865_0001_1_00, eventCount=1
2016-01-14 18:31:28,116 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:28,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EE8E299B3EA54ED4BD22969EB6C0C20B, taskAttemptId=attempt_1452796287865_0001_1_00_000000_0, startTime=1452796288062, finishTime=1452796288116, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288107, Process_Duration=3, Process_End_Time=1452796288110, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:28,116 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287865_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:28,117 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:28,117 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796287865_0001_1_00_000000_0
2016-01-14 18:31:28,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:TASK_FINISHED]: vertexName=EE8E299B3EA54ED4BD22969EB6C0C20B, taskId=task_1452796287865_0001_1_00_000000, startTime=1452796288062, finishTime=1452796288117, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1452796287865_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288107, Process_Duration=3, Process_End_Time=1452796288110, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:28,117 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287865_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:28,118 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B] : 1
2016-01-14 18:31:28,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:28,118 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B]
2016-01-14 18:31:28,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:28,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:VERTEX_FINISHED]: vertexName=EE8E299B3EA54ED4BD22969EB6C0C20B, vertexId=vertex_1452796287865_0001_1_00, initRequestedTime=1452796288035, initedTime=1452796288048, startRequestedTime=1452796288049, startedTime=1452796288049, finishTime=1452796288118, timeTaken=69, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288107, Process_Duration=3, Process_End_Time=1452796288110, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796288062, firstTasksToStart=[ task_1452796287865_0001_1_00_000000 ], lastTaskFinishTime=1452796288116, lastTasksToFinish=[ task_1452796287865_0001_1_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796287865_0001_1_00_000000 ], longestDurationTasks=[ task_1452796287865_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:28,119 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:28,119 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:28,120 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA] attempt: attempt_1452796287865_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:28,120 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E427BD1C382A4B65824E761E214915DA
2016-01-14 18:31:28,120 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA]
2016-01-14 18:31:28,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796287865_0001_00_000001, stoppedTime=1452796288120, exitStatus=0
2016-01-14 18:31:28,120 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:28,120 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:28,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:28,121 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:28,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796288107, Process_Duration=3, Process_End_Time=1452796288110, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:28,121 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:TASK_STARTED]: vertexName=E427BD1C382A4B65824E761E214915DA, taskId=task_1452796287865_0001_1_01_000000, scheduledTime=1452796288121, launchTime=1452796288121
2016-01-14 18:31:28,121 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:28,121 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,122 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287865_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:28,122 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA]
2016-01-14 18:31:28,122 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796287865_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:28,122 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: E427BD1C382A4B65824E761E214915DA, VertexParallelism: 1, TaskAttemptID:attempt_1452796287865_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EE8E299B3EA54ED4BD22969EB6C0C20B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BD592F6D3A2A4610828933711FC6035B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:28,122 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287865_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:28,122 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:28,122 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:28,122 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796287865_0001_00_000001
2016-01-14 18:31:28,123 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796287865_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:28,123 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796287865_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:28,123 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796287865_0001_00_000002
2016-01-14 18:31:28,124 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:28,124 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796287865_0001_00_000001, Exiting
2016-01-14 18:31:28,124 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796287865_0001_00_000001
2016-01-14 18:31:28,124 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796287865_0001_00_000001
2016-01-14 18:31:28,125 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796287865_0001_00_000002
2016-01-14 18:31:28,126 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796287865_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:28,126 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:28,126 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796287865_0001_00_000001 completed successfully
2016-01-14 18:31:28,127 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:28,128 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:28,128 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:28,128 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796287865_0001_00_000002, launchTime=1452796288128
2016-01-14 18:31:28,129 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796287865_0001_00_000002
2016-01-14 18:31:28,129 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796287865_0001_1_01_000000_0] to container: [container_1452796287865_0001_00_000002]
2016-01-14 18:31:28,129 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:28,130 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796287865_0001_1_01_000000_0] started. Is using containerId: [container_1452796287865_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:28,130 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E427BD1C382A4B65824E761E214915DA, taskAttemptId=attempt_1452796287865_0001_1_01_000000_0, startTime=1452796288129, containerId=container_1452796287865_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796287865_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:28,130 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:28,130 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796287865_0001_00_000002 given task: attempt_1452796287865_0001_1_01_000000_0
2016-01-14 18:31:28,130 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287865_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:28,130 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287865_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:28,130 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796287865_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796287865_0001_1_01_000000_0
2016-01-14 18:31:28 Starting to run new task attempt: attempt_1452796287865_0001_1_01_000000_0
2016-01-14 18:31:28,131 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796287865_0001_1_01_000000_0
2016-01-14 18:31:28,131 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,131 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:28,131 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:28,131 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:28,131 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:28,131 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:28,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB] every/(1/1) ...mblyplatform/0_'field'_-1, VertexName: E427BD1C382A4B65824E761E214915DA, VertexParallelism: 1, TaskAttemptID:attempt_1452796287865_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=EE8E299B3EA54ED4BD22969EB6C0C20B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BD592F6D3A2A4610828933711FC6035B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:28,132 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:28,132 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796287865_0001_1_01_000000_0
2016-01-14 18:31:28,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:28,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,133 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796287865_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:28,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BD592F6D3A2A4610828933711FC6035B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:28,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=EE8E299B3EA54ED4BD22969EB6C0C20B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:28,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:28,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: EE8E299B3EA54ED4BD22969EB6C0C20B
2016-01-14 18:31:28,137 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:28,137 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:28,137 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:28,137 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: EE8E299B3EA54ED4BD22969EB6C0C20B
2016-01-14 18:31:28,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: EE8E299B3EA54ED4BD22969EB6C0C20B
2016-01-14 18:31:28,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:28,141 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BD592F6D3A2A4610828933711FC6035B
2016-01-14 18:31:28,141 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:28,141 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: BD592F6D3A2A4610828933711FC6035B
2016-01-14 18:31:28,142 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E427BD1C382A4B65824E761E214915DA, ordinal: 1
2016-01-14 18:31:28,142 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E427BD1C382A4B65824E761E214915DA, mem on start (mb), free: 210, total: 369, max: 672
2016-01-14 18:31:28,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:28,142 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:28,150 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:28,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BD592F6D3A2A4610828933711FC6035B
2016-01-14 18:31:28,151 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:28,151 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:28,151 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:28,151 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:28,151 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:28,151 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:28,151 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:28,151 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, EE8E299B3EA54ED4BD22969EB6C0C20B, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:28,151 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BD592F6D3A2A4610828933711FC6035B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:28,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:28,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: EE8E299B3EA54ED4BD22969EB6C0C20B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:28,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:28,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:28,152 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: EE8E299B3EA54ED4BD22969EB6C0C20B
2016-01-14 18:31:28,153 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:28,153 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:28,153 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: EE8E299B3EA54ED4BD22969EB6C0C20B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:28,153 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:28,153 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:28,153 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:28,154 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: EE8E299B3EA54ED4BD22969EB6C0C20B
2016-01-14 18:31:28,154 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:28,155 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796287865_0001_1_01_000000_0
2016-01-14 18:31:28,158 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:28,159 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C941BF295D4447B1B25964CF92D5ECB1
2016-01-14 18:31:28,159 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1"], id: BD592F6D3A2A4610828933711FC6035B
2016-01-14 18:31:28,159 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_-1"]
2016-01-14 18:31:28,159 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C941BF295D4447B1B25964CF92D5ECB1, for 1 inputs
2016-01-14 18:31:28,159 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796287865_0001_1_00_000000_0_10117, runDuration: 43614, ]
2016-01-14 18:31:28,160 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796287865_0001_1_00_000000_0_10117] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796288160, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:28,160 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : EE8E299B3EA54ED4BD22969EB6C0C20B
2016-01-14 18:31:28,161 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:28,161 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [EE8E299B3EA54ED4BD22969EB6C0C20B] #20 in 1ms
2016-01-14 18:31:28,161 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [EE8E299B3EA54ED4BD22969EB6C0C20B] #20, status:false, isInterrupted:false
2016-01-14 18:31:28,163 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:28,163 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:28,163 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:28,163 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:28,163 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:28,164 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : EE8E299B3EA54ED4BD22969EB6C0C20B
2016-01-14 18:31:28,164 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:28,164 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E427BD1C382A4B65824E761E214915DA, all 1 inputs ready in: 00:00:00.005
2016-01-14 18:31:28,165 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@26de5f2c; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@513a55a3
2016-01-14 18:31:28,169 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:28,170 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:28,170 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796287865_0001_1_01_000000_0
2016-01-14 18:31:28,170 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796287865_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:28,171 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796287865_0001_1_01_000000_0
2016-01-14 18:31:28,171 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E427BD1C382A4B65824E761E214915DA, mem on close (mb), free: 206, total: 369, max: 672
2016-01-14 18:31:28,171 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: EE8E299B3EA54ED4BD22969EB6C0C20B
2016-01-14 18:31:28,172 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:28,172 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:28,172 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:28,172 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:28,172 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:28 Completed running task attempt: attempt_1452796287865_0001_1_01_000000_0
2016-01-14 18:31:28,173 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796287865_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:28,174 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796288164, Process_Duration=7, Process_End_Time=1452796288171, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:28,174 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:28,174 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:28,174 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E427BD1C382A4B65824E761E214915DA, taskAttemptId=attempt_1452796287865_0001_1_01_000000_0, startTime=1452796288129, finishTime=1452796288174, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288164, Process_Duration=7, Process_End_Time=1452796288171, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:28,175 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796287865_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:28,175 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:28,175 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,175 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796287865_0001_1_01_000000_0
2016-01-14 18:31:28,175 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796287865_0001_00_000002
2016-01-14 18:31:28,175 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:28,176 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:TASK_FINISHED]: vertexName=E427BD1C382A4B65824E761E214915DA, taskId=task_1452796287865_0001_1_01_000000, startTime=1452796288129, finishTime=1452796288175, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1452796287865_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288164, Process_Duration=7, Process_End_Time=1452796288171, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:28,176 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796287865_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:28,176 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA] : 1
2016-01-14 18:31:28,176 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA]
2016-01-14 18:31:28,177 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:28,177 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:VERTEX_FINISHED]: vertexName=E427BD1C382A4B65824E761E214915DA, vertexId=vertex_1452796287865_0001_1_01, initRequestedTime=1452796288037, initedTime=1452796288046, startRequestedTime=1452796288051, startedTime=1452796288051, finishTime=1452796288176, timeTaken=125, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288164, Process_Duration=7, Process_End_Time=1452796288171, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796288129, firstTasksToStart=[ task_1452796287865_0001_1_01_000000 ], lastTaskFinishTime=1452796288174, lastTasksToFinish=[ task_1452796287865_0001_1_01_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796287865_0001_1_01_000000 ], longestDurationTasks=[ task_1452796287865_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:28,177 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:28,177 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:28,178 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:28,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796287865_0001_00_000002, stoppedTime=1452796288178, exitStatus=0
2016-01-14 18:31:28,178 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:28,178 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:28,178 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796287865_0001_1
2016-01-14 18:31:28,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796287865_0001_1
2016-01-14 18:31:28,178 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: BD592F6D3A2A4610828933711FC6035B for vertex: vertex_1452796287865_0001_1_01 [E427BD1C382A4B65824E761E214915DA]
2016-01-14 18:31:28,186 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796287865_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:28,186 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796287865_0001_00_000002: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:28,186 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796287865_0001_00_000002, Exiting
2016-01-14 18:31:28,186 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796287865_0001_00_000002
2016-01-14 18:31:28,186 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796287865_0001_00_000002
2016-01-14 18:31:28,186 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796287865_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:28,186 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796287865_0001_00_000002 completed successfully
2016-01-14 18:31:28,189 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796287865_0001_1_00 [EE8E299B3EA54ED4BD22969EB6C0C20B]
2016-01-14 18:31:28,193 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796287865_0001_1, queueSize=0
2016-01-14 18:31:28,193 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:28,193 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796287865_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796287865_0001_1, startTime=1452796288034, finishTime=1452796288190, timeTaken=156, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=773849088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592576271, Process_Duration=10, Process_End_Time=2905592576281, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:28,193 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796287865_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:28,193 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796287865_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:28,194 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796287865_0001_00_000002
2016-01-14 18:31:28,194 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796287865_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:28 Completed Dag: dag_1452796287865_0001_1
2016-01-14 18:31:28,196 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796287865_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:28,196 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796287865_0001_1_post
2016-01-14 18:31:28,198 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,199 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:28,201 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_'field'_-1, applicationId=application_1452796287865_0001
2016-01-14 18:31:28,202 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:28,203 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:28,203 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:28,203 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:28,203 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:28,203 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:28,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:28,204 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:28,204 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:28,205 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:28,205 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:28,205 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:28,206 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:28,211 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/662A195BDF2649B8A70C07216DC792CC/BC2862B3D53A4EF499C98443287B18FB/.tez/application_1452796287865_0001
2016-01-14 18:31:28,212 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.375
2016-01-14 18:31:28,283 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_-1
2016-01-14 18:31:28,302 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:28,302 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:28,311 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:28,312 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:28,349 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 18:31:28,356 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 18:31:28,357 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:28,357 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:28,357 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:28,372 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:28,378 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'
2016-01-14 18:31:28,427 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:28,428 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:28,428 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'"]
2016-01-14 18:31:28,428 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:28,429 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:28,429 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:28,431 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/0_'field'_'field'
2016-01-14 18:31:28,431 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:28,454 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:28,459 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:28,460 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:28,460 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:28,471 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459/.tez/application_1452796288460_0001 doesn't exist and is created
2016-01-14 18:31:28,491 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:28,491 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:28,492 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459/.tez/application_1452796288460_0001
2016-01-14 18:31:28,494 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796288460_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:28,511 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:28,511 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:28,513 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:28,513 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:28,513 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459/.tez/application_1452796288460_0001/localmode-log-dir
2016-01-14 18:31:28,513 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459/.tez/application_1452796288460_0001/localmode-log-dir/history.txt.appattempt_1452796288460_0001_000000, maxErrors=10
2016-01-14 18:31:28,513 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:28,514 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796288460_0001_000000, appSubmitTime=1452796288493, launchTime=1452796288493
2016-01-14 18:31:28,519 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:57813
2016-01-14 18:31:28,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:28,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:28,527 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:28,527 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:28,528 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:28,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796288460_0001_000000, startTime=1452796288530
2016-01-14 18:31:28,531 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:28,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:28,592 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:28,592 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:28,592 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1452796288460_0001, dagName=[6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459] every/(1/1) ...latform/0_'field'_'field'
2016-01-14 18:31:28,617 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459] every/(1/1) ...latform/0_'field'_'field'
2016-01-14 18:31:28,618 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796288460_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459/.tez/application_1452796288460_0001/localmode-log-dir/dag_1452796288460_0001_1.dot
2016-01-14 18:31:28,619 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459/.tez/application_1452796288460_0001/localmode-log-dir/dag_1452796288460_0001_1-tez-dag.pb.txt
2016-01-14 18:31:28,625 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796288460_0001_1
2016-01-14 18:31:28,625 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,625 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459] every/(1/1) ...latform/0_'field'_'field'
2016-01-14 18:31:28 Running Dag: dag_1452796288460_0001_1
2016-01-14 18:31:28,634 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796288460_0001_1, submitTime=1452796288617
2016-01-14 18:31:28,634 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:28,635 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BD4F80BF78C64CC4A1D6F86A1DF708D3
2016-01-14 18:31:28,637 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:28,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796288460_0001_1, initTime=1452796288634
2016-01-14 18:31:28,637 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796288460_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:28,637 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:28,638 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:DAG_STARTED]: dagID=dag_1452796288460_0001_1, startTime=1452796288637
2016-01-14 18:31:28,638 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:28,638 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796288460_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:28,639 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C8B223B8DBF2410FA3C6CD3915C32335 : {8B03DB6D33CD4B46BAF04B50F6EA6FD4={InputName=8B03DB6D33CD4B46BAF04B50F6EA6FD4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:28,639 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8B03DB6D33CD4B46BAF04B50F6EA6FD4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:28,639 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335]
2016-01-14 18:31:28,639 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335]
2016-01-14 18:31:28,639 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335]
2016-01-14 18:31:28,639 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335]. Starting root input initializers: 1
2016-01-14 18:31:28,640 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:28,640 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3]
2016-01-14 18:31:28,642 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1452796288460_0001, dagName=[6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459] every/(1/1) ...latform/0_'field'_'field'
2016-01-14 18:31:28,643 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:28,643 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3]
2016-01-14 18:31:28,643 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3]
2016-01-14 18:31:28,644 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3]
2016-01-14 18:31:28,644 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=01FB21A340B84C7CAB9511953F12EE78, vertexId=vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:28,644 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=01FB21A340B84C7CAB9511953F12EE78, vertexId=vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3]
2016-01-14 18:31:28,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:28,646 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8B03DB6D33CD4B46BAF04B50F6EA6FD4 on vertex vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335]
2016-01-14 18:31:28,646 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=01FB21A340B84C7CAB9511953F12EE78, vertexName=BD4F80BF78C64CC4A1D6F86A1DF708D3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:28,646 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=01FB21A340B84C7CAB9511953F12EE78, vertexId=vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3]
2016-01-14 18:31:28,647 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:28,647 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BD4F80BF78C64CC4A1D6F86A1DF708D3, vertexId=vertex_1452796288460_0001_1_01, initRequestedTime=1452796288640, initedTime=1452796288647, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:28,648 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:28,649 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:28,649 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8B03DB6D33CD4B46BAF04B50F6EA6FD4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:28,649 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:28,650 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:28,652 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:28,652 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8B03DB6D33CD4B46BAF04B50F6EA6FD4 on vertex vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335]
2016-01-14 18:31:28,652 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:28,653 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8B03DB6D33CD4B46BAF04B50F6EA6FD4=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:28,653 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335] parallelism set to 1
2016-01-14 18:31:28,653 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:28,653 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796288460_0001_1_00, eventCount=1
2016-01-14 18:31:28,654 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:28,654 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C8B223B8DBF2410FA3C6CD3915C32335, vertexId=vertex_1452796288460_0001_1_00, initRequestedTime=1452796288638, initedTime=1452796288653, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:28,654 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335] with distanceFromRoot: 0
2016-01-14 18:31:28,654 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:28,654 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C8B223B8DBF2410FA3C6CD3915C32335
2016-01-14 18:31:28,655 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335]
2016-01-14 18:31:28,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:28,655 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796288460_0001_1_00, startRequestedTime=1452796288654, startedTime=1452796288654
2016-01-14 18:31:28,655 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:28,656 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:28,656 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:TASK_STARTED]: vertexName=C8B223B8DBF2410FA3C6CD3915C32335, taskId=task_1452796288460_0001_1_00_000000, scheduledTime=1452796288656, launchTime=1452796288656
2016-01-14 18:31:28,656 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796288460_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:28,657 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796288460_0001_1_00 for vertex: vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3] numStartedSources: 1 numSources: 1
2016-01-14 18:31:28,657 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3] with distanceFromRoot: 1
2016-01-14 18:31:28,658 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335]
2016-01-14 18:31:28,658 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C8B223B8DBF2410FA3C6CD3915C32335 in vertex: BD4F80BF78C64CC4A1D6F86A1DF708D3
2016-01-14 18:31:28,658 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BD4F80BF78C64CC4A1D6F86A1DF708D3 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:28,659 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:28,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796288460_0001_1_01, startRequestedTime=1452796288658, startedTime=1452796288658
2016-01-14 18:31:28,659 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:28,659 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459] every/(1/1) ...latform/0_'field'_'field', VertexName: C8B223B8DBF2410FA3C6CD3915C32335, VertexParallelism: 1, TaskAttemptID:attempt_1452796288460_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8B03DB6D33CD4B46BAF04B50F6EA6FD4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD4F80BF78C64CC4A1D6F86A1DF708D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:28,659 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796288460_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:28,659 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:28,660 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:28,660 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:28,660 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:28,661 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796288460_0001_00_000001
2016-01-14 18:31:28,661 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:28,663 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796288460_0001_00_000001
2016-01-14 18:31:28,663 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:28,664 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:28,664 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:28,664 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796288460_0001_00_000001, launchTime=1452796288664
2016-01-14 18:31:28,665 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796288460_0001_00_000001
2016-01-14 18:31:28,665 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796288460_0001_1_00_000000_0] to container: [container_1452796288460_0001_00_000001]
2016-01-14 18:31:28,665 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:28,666 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796288460_0001_1_00_000000_0] started. Is using containerId: [container_1452796288460_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:28,666 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C8B223B8DBF2410FA3C6CD3915C32335, taskAttemptId=attempt_1452796288460_0001_1_00_000000_0, startTime=1452796288666, containerId=container_1452796288460_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796288460_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:28,666 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796288460_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:28,666 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796288460_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:28,667 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:28,667 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796288460_0001_00_000001 given task: attempt_1452796288460_0001_1_00_000000_0
2016-01-14 18:31:28,667 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796288460_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796288460_0001_1_00_000000_0
2016-01-14 18:31:28 Starting to run new task attempt: attempt_1452796288460_0001_1_00_000000_0
2016-01-14 18:31:28,668 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796288460_0001_1_00_000000_0
2016-01-14 18:31:28,668 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,668 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:28,668 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:28,668 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:28,668 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:28,669 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:28,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459] every/(1/1) ...latform/0_'field'_'field', VertexName: C8B223B8DBF2410FA3C6CD3915C32335, VertexParallelism: 1, TaskAttemptID:attempt_1452796288460_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8B03DB6D33CD4B46BAF04B50F6EA6FD4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD4F80BF78C64CC4A1D6F86A1DF708D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:28,669 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:28,670 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796288460_0001_1_00_000000_0
2016-01-14 18:31:28,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:28,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,670 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796288460_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:28,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,673 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796288460_0001, with dag id: dag_1452796288460_0001_1
2016-01-14 18:31:28,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8B03DB6D33CD4B46BAF04B50F6EA6FD4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:28,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:28,674 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8B03DB6D33CD4B46BAF04B50F6EA6FD4
2016-01-14 18:31:28,674 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8B03DB6D33CD4B46BAF04B50F6EA6FD4
2016-01-14 18:31:28,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BD4F80BF78C64CC4A1D6F86A1DF708D3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:28,675 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:28,676 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:28,676 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8B03DB6D33CD4B46BAF04B50F6EA6FD4
2016-01-14 18:31:28,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8B03DB6D33CD4B46BAF04B50F6EA6FD4
2016-01-14 18:31:28,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:28,677 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BD4F80BF78C64CC4A1D6F86A1DF708D3
2016-01-14 18:31:28,679 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:28,679 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BD4F80BF78C64CC4A1D6F86A1DF708D3
2016-01-14 18:31:28,679 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BD4F80BF78C64CC4A1D6F86A1DF708D3
2016-01-14 18:31:28,681 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C8B223B8DBF2410FA3C6CD3915C32335, ordinal: 0
2016-01-14 18:31:28,681 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8B223B8DBF2410FA3C6CD3915C32335, mem on start (mb), free: 310, total: 368, max: 672
2016-01-14 18:31:28,681 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,681 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:28,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:28,682 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:28,682 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:28,682 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:28,682 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:28,683 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:28,683 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:28,683 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:28,683 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8B03DB6D33CD4B46BAF04B50F6EA6FD4, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:28,683 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BD4F80BF78C64CC4A1D6F86A1DF708D3, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:28,683 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:28,684 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8B03DB6D33CD4B46BAF04B50F6EA6FD4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:28,684 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8B03DB6D33CD4B46BAF04B50F6EA6FD4
2016-01-14 18:31:28,684 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:28,684 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8B03DB6D33CD4B46BAF04B50F6EA6FD4
2016-01-14 18:31:28,684 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:28,684 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:28,685 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:28,685 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796288460_0001_1_00_000000_0
2016-01-14 18:31:28,685 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:28,686 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@71820ab2
2016-01-14 18:31:28,686 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@71820ab2
2016-01-14 18:31:28,687 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:28,687 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:28,689 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:28,690 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 8B03DB6D33CD4B46BAF04B50F6EA6FD4
2016-01-14 18:31:28,690 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 896E6FF5E47D4ED3AA8441468882B469
2016-01-14 18:31:28,690 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 896E6FF5E47D4ED3AA8441468882B469
2016-01-14 18:31:28,691 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@5510808f; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2e6857ea; comparator=cascading.tuple.hadoop.util.TupleComparator@4463db73; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:28,691 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:28,691 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:28,691 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:28,705 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:28,706 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:28,706 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:28,706 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:28,707 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:28,707 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:28,708 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:28,708 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C8B223B8DBF2410FA3C6CD3915C32335, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:28,711 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796288460_0001_1_00_000000_0
2016-01-14 18:31:28,711 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C8B223B8DBF2410FA3C6CD3915C32335, mem on close (mb), free: 208, total: 368, max: 672
2016-01-14 18:31:28,711 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:28,711 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:28,711 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:28,712 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:28,714 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:28 Completed running task attempt: attempt_1452796288460_0001_1_00_000000_0
2016-01-14 18:31:28,716 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796288460_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:28,717 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796288460_0001_1_00, eventCount=1
2016-01-14 18:31:28,717 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C8B223B8DBF2410FA3C6CD3915C32335, taskAttemptId=attempt_1452796288460_0001_1_00_000000_0, startTime=1452796288666, finishTime=1452796288717, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288708, Process_Duration=3, Process_End_Time=1452796288711, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:28,717 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796288460_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:28,717 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796288460_0001_1_00_000000_0
2016-01-14 18:31:28,718 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:TASK_FINISHED]: vertexName=C8B223B8DBF2410FA3C6CD3915C32335, taskId=task_1452796288460_0001_1_00_000000, startTime=1452796288666, finishTime=1452796288717, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1452796288460_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288708, Process_Duration=3, Process_End_Time=1452796288711, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:28,718 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796288460_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:28,718 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:28,718 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:28,718 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335] : 1
2016-01-14 18:31:28,718 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335]
2016-01-14 18:31:28,718 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:28,721 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:28,721 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:VERTEX_FINISHED]: vertexName=C8B223B8DBF2410FA3C6CD3915C32335, vertexId=vertex_1452796288460_0001_1_00, initRequestedTime=1452796288638, initedTime=1452796288653, startRequestedTime=1452796288654, startedTime=1452796288654, finishTime=1452796288718, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288708, Process_Duration=3, Process_End_Time=1452796288711, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796288666, firstTasksToStart=[ task_1452796288460_0001_1_00_000000 ], lastTaskFinishTime=1452796288717, lastTasksToFinish=[ task_1452796288460_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796288460_0001_1_00_000000 ], longestDurationTasks=[ task_1452796288460_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:28,721 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:28,722 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3] attempt: attempt_1452796288460_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:28,722 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BD4F80BF78C64CC4A1D6F86A1DF708D3
2016-01-14 18:31:28,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796288708, Process_Duration=3, Process_End_Time=1452796288711, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:28,722 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:28,722 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,722 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3]
2016-01-14 18:31:28,722 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:28,723 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796288460_0001_00_000001, stoppedTime=1452796288723, exitStatus=0
2016-01-14 18:31:28,723 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:28,723 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:28,723 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:28,724 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796288460_0001_00_000001
2016-01-14 18:31:28,724 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796288460_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:28,724 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796288460_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:28,724 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796288460_0001_00_000001, Exiting
2016-01-14 18:31:28,724 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796288460_0001_00_000001
2016-01-14 18:31:28,724 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796288460_0001_00_000001
2016-01-14 18:31:28,724 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:TASK_STARTED]: vertexName=BD4F80BF78C64CC4A1D6F86A1DF708D3, taskId=task_1452796288460_0001_1_01_000000, scheduledTime=1452796288724, launchTime=1452796288724
2016-01-14 18:31:28,725 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796288460_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:28,724 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:28,725 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796288460_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:28,725 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3]
2016-01-14 18:31:28,725 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:28,725 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459] every/(1/1) ...latform/0_'field'_'field', VertexName: BD4F80BF78C64CC4A1D6F86A1DF708D3, VertexParallelism: 1, TaskAttemptID:attempt_1452796288460_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8B223B8DBF2410FA3C6CD3915C32335, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=01FB21A340B84C7CAB9511953F12EE78, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:28,725 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796288460_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:28,725 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:28,726 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796288460_0001_00_000002
2016-01-14 18:31:28,726 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:28,727 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796288460_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:28,727 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796288460_0001_00_000001 completed successfully
2016-01-14 18:31:28,727 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:28,727 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796288460_0001_00_000002
2016-01-14 18:31:28,728 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:28,730 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:28,730 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:28,731 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796288460_0001_00_000002, launchTime=1452796288730
2016-01-14 18:31:28,731 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796288460_0001_00_000002
2016-01-14 18:31:28,732 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796288460_0001_1_01_000000_0] to container: [container_1452796288460_0001_00_000002]
2016-01-14 18:31:28,732 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:28,732 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796288460_0001_00_000002 given task: attempt_1452796288460_0001_1_01_000000_0
2016-01-14 18:31:28,732 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796288460_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796288460_0001_1_01_000000_0
2016-01-14 18:31:28,732 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796288460_0001_1_01_000000_0] started. Is using containerId: [container_1452796288460_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:28,733 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BD4F80BF78C64CC4A1D6F86A1DF708D3, taskAttemptId=attempt_1452796288460_0001_1_01_000000_0, startTime=1452796288732, containerId=container_1452796288460_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796288460_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:28,733 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:28,733 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796288460_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:28,733 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796288460_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:28 Starting to run new task attempt: attempt_1452796288460_0001_1_01_000000_0
2016-01-14 18:31:28,734 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796288460_0001_1_01_000000_0
2016-01-14 18:31:28,734 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,734 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:28,735 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:28,735 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:28,735 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:28,735 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:28,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459] every/(1/1) ...latform/0_'field'_'field', VertexName: BD4F80BF78C64CC4A1D6F86A1DF708D3, VertexParallelism: 1, TaskAttemptID:attempt_1452796288460_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C8B223B8DBF2410FA3C6CD3915C32335, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=01FB21A340B84C7CAB9511953F12EE78, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:28,736 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:28,736 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796288460_0001_1_01_000000_0
2016-01-14 18:31:28,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:28,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,737 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796288460_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:28,737 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C8B223B8DBF2410FA3C6CD3915C32335, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:28,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=01FB21A340B84C7CAB9511953F12EE78, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:28,738 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:28,739 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 01FB21A340B84C7CAB9511953F12EE78
2016-01-14 18:31:28,739 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:28,739 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 01FB21A340B84C7CAB9511953F12EE78
2016-01-14 18:31:28,741 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:28,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:28,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C8B223B8DBF2410FA3C6CD3915C32335
2016-01-14 18:31:28,744 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:28,745 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:28,745 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C8B223B8DBF2410FA3C6CD3915C32335
2016-01-14 18:31:28,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C8B223B8DBF2410FA3C6CD3915C32335
2016-01-14 18:31:28,749 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BD4F80BF78C64CC4A1D6F86A1DF708D3, ordinal: 1
2016-01-14 18:31:28,749 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD4F80BF78C64CC4A1D6F86A1DF708D3, mem on start (mb), free: 203, total: 368, max: 672
2016-01-14 18:31:28,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:28,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:28,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:28,751 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:28,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 01FB21A340B84C7CAB9511953F12EE78
2016-01-14 18:31:28,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:28,751 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:28,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:28,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:28,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:28,753 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:28,754 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:28,755 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 01FB21A340B84C7CAB9511953F12EE78, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:28,755 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C8B223B8DBF2410FA3C6CD3915C32335, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:28,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:28,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C8B223B8DBF2410FA3C6CD3915C32335
2016-01-14 18:31:28,755 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:28,756 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:28,756 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C8B223B8DBF2410FA3C6CD3915C32335 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:28,756 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:28,756 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:28,756 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:28,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C8B223B8DBF2410FA3C6CD3915C32335 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:28,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:28,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:28,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C8B223B8DBF2410FA3C6CD3915C32335
2016-01-14 18:31:28,757 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:28,758 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796288460_0001_1_01_000000_0
2016-01-14 18:31:28,758 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796288460_0001_1_00_000000_0_10123, runDuration: 37745, ]
2016-01-14 18:31:28,760 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:28,761 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 896E6FF5E47D4ED3AA8441468882B469
2016-01-14 18:31:28,761 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'"], id: 01FB21A340B84C7CAB9511953F12EE78
2016-01-14 18:31:28,761 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_'field'"]
2016-01-14 18:31:28,761 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 896E6FF5E47D4ED3AA8441468882B469, for 1 inputs
2016-01-14 18:31:28,762 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796288460_0001_1_00_000000_0_10123] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796288762, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:28,762 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C8B223B8DBF2410FA3C6CD3915C32335
2016-01-14 18:31:28,762 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:28,763 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C8B223B8DBF2410FA3C6CD3915C32335] #21 in 1ms
2016-01-14 18:31:28,763 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C8B223B8DBF2410FA3C6CD3915C32335] #21, status:false, isInterrupted:false
2016-01-14 18:31:28,763 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:28,763 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:28,763 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:28,764 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:28,764 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:28,764 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C8B223B8DBF2410FA3C6CD3915C32335
2016-01-14 18:31:28,764 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BD4F80BF78C64CC4A1D6F86A1DF708D3, all 1 inputs ready in: 00:00:00.002
2016-01-14 18:31:28,764 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:28,765 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@84644a2; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3d43ebec
2016-01-14 18:31:28,768 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:28,768 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:28,768 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796288460_0001_1_01_000000_0
2016-01-14 18:31:28,768 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796288460_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:28,769 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796288460_0001_1_01_000000_0
2016-01-14 18:31:28,769 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD4F80BF78C64CC4A1D6F86A1DF708D3, mem on close (mb), free: 199, total: 368, max: 672
2016-01-14 18:31:28,769 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C8B223B8DBF2410FA3C6CD3915C32335
2016-01-14 18:31:28,769 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:28,770 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:28,770 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:28,770 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:28,770 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:28 Completed running task attempt: attempt_1452796288460_0001_1_01_000000_0
2016-01-14 18:31:28,771 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796288460_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:28,771 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:28,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BD4F80BF78C64CC4A1D6F86A1DF708D3, taskAttemptId=attempt_1452796288460_0001_1_01_000000_0, startTime=1452796288732, finishTime=1452796288771, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288764, Process_Duration=5, Process_End_Time=1452796288769, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:28,771 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796288460_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:28,772 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796288460_0001_1_01_000000_0
2016-01-14 18:31:28,772 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:28,772 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:TASK_FINISHED]: vertexName=BD4F80BF78C64CC4A1D6F86A1DF708D3, taskId=task_1452796288460_0001_1_01_000000, startTime=1452796288732, finishTime=1452796288772, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1452796288460_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288764, Process_Duration=5, Process_End_Time=1452796288769, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:28,772 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796288460_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:28,773 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3] : 1
2016-01-14 18:31:28,773 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3]
2016-01-14 18:31:28,774 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:VERTEX_FINISHED]: vertexName=BD4F80BF78C64CC4A1D6F86A1DF708D3, vertexId=vertex_1452796288460_0001_1_01, initRequestedTime=1452796288640, initedTime=1452796288647, startRequestedTime=1452796288658, startedTime=1452796288658, finishTime=1452796288773, timeTaken=115, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796288764, Process_Duration=5, Process_End_Time=1452796288769, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796288732, firstTasksToStart=[ task_1452796288460_0001_1_01_000000 ], lastTaskFinishTime=1452796288771, lastTasksToFinish=[ task_1452796288460_0001_1_01_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796288460_0001_1_01_000000 ], longestDurationTasks=[ task_1452796288460_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:28,774 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:28,774 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:28,775 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796288460_0001_00_000002, stoppedTime=1452796288774, exitStatus=0
2016-01-14 18:31:28,775 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:28,775 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=385875968, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796288764, Process_Duration=5, Process_End_Time=1452796288769, Read_Duration=3, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:28,775 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:28,775 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,775 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:28,776 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796288460_0001_1
2016-01-14 18:31:28,776 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796288460_0001_1
2016-01-14 18:31:28,776 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796288460_0001_1_00 [C8B223B8DBF2410FA3C6CD3915C32335]
2016-01-14 18:31:28,774 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:28,776 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796288460_0001_00_000002
2016-01-14 18:31:28,776 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 01FB21A340B84C7CAB9511953F12EE78 for vertex: vertex_1452796288460_0001_1_01 [BD4F80BF78C64CC4A1D6F86A1DF708D3]
2016-01-14 18:31:28,777 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:28,778 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:28,780 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796288460_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:28,780 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796288460_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:28,781 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796288460_0001_00_000002, Exiting
2016-01-14 18:31:28,781 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796288460_0001_00_000002
2016-01-14 18:31:28,781 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796288460_0001_00_000002
2016-01-14 18:31:28,781 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796288460_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:28,781 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796288460_0001_00_000002 completed successfully
2016-01-14 18:31:28,786 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796288460_0001_1, queueSize=0
2016-01-14 18:31:28,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:28,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796288460_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796288460_0001_1, startTime=1452796288637, finishTime=1452796288784, timeTaken=147, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=771751936, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592577472, Process_Duration=8, Process_End_Time=2905592577480, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:28,787 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796288460_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:28,787 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796288460_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:28,787 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796288460_0001_00_000002
2016-01-14 18:31:28,788 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796288460_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:28 Completed Dag: dag_1452796288460_0001_1
2016-01-14 18:31:28,789 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796288460_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:28,789 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796288460_0001_1_post
2016-01-14 18:31:28,789 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:28,789 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:28,794 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_'field'_'field', applicationId=application_1452796288460_0001
2016-01-14 18:31:28,795 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:28,795 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:28,795 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:28,795 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:28,795 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:28,796 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:28,796 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:28,797 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:28,797 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:28,797 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:28,797 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:28,798 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:28,798 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:28,804 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6625D52ADA9C4471B3D2231A169EEBCE/D5C6E7A8BFF040E0BBA4F36F6B662459/.tez/application_1452796288460_0001
2016-01-14 18:31:28,804 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.377
2016-01-14 18:31:28,820 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:28,820 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:28,873 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_'field'
2016-01-14 18:31:28,898 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:28,898 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:28,944 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 18:31:28,956 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 18:31:28,957 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:28,957 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:28,958 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:28,975 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:28,981 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS
2016-01-14 18:31:29,029 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:29,029 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:29,029 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS"]
2016-01-14 18:31:29,029 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:29,029 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:29,029 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:29,030 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/0_'field'_RESULTS
2016-01-14 18:31:29,030 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:29,043 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:29,047 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:29,048 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:29,048 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:29,057 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610/.tez/application_1452796289047_0001 doesn't exist and is created
2016-01-14 18:31:29,071 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:29,071 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:29,071 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610/.tez/application_1452796289047_0001
2016-01-14 18:31:29,073 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796289047_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:29,083 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:29,084 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:29,085 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:29,085 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:29,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610/.tez/application_1452796289047_0001/localmode-log-dir
2016-01-14 18:31:29,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610/.tez/application_1452796289047_0001/localmode-log-dir/history.txt.appattempt_1452796289047_0001_000000, maxErrors=10
2016-01-14 18:31:29,086 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:29,086 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796289047_0001_000000, appSubmitTime=1452796289072, launchTime=1452796289072
2016-01-14 18:31:29,090 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:29,091 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:46049
2016-01-14 18:31:29,092 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:29,093 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:29,093 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:29,094 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:29,095 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796289047_0001_000000, startTime=1452796289095
2016-01-14 18:31:29,095 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:29,095 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:29,171 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:29,171 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:29,171 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1452796289047_0001, dagName=[3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610] every/(1/1) ...latform/0_'field'_RESULTS
2016-01-14 18:31:29,197 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610] every/(1/1) ...latform/0_'field'_RESULTS
2016-01-14 18:31:29,198 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796289047_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610/.tez/application_1452796289047_0001/localmode-log-dir/dag_1452796289047_0001_1.dot
2016-01-14 18:31:29,199 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610/.tez/application_1452796289047_0001/localmode-log-dir/dag_1452796289047_0001_1-tez-dag.pb.txt
2016-01-14 18:31:29,205 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796289047_0001_1
2016-01-14 18:31:29,205 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,205 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610] every/(1/1) ...latform/0_'field'_RESULTS
2016-01-14 18:31:29 Running Dag: dag_1452796289047_0001_1
2016-01-14 18:31:29,214 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796289047_0001_1, submitTime=1452796289197
2016-01-14 18:31:29,214 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:29,215 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AA68ED39F0E0454291D3821BD0978850
2016-01-14 18:31:29,216 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:29,216 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796289047_0001_1, initTime=1452796289215
2016-01-14 18:31:29,216 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:29,216 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796289047_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:29,217 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:DAG_STARTED]: dagID=dag_1452796289047_0001_1, startTime=1452796289217
2016-01-14 18:31:29,217 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:29,217 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796289047_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:29,217 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DD775E21C6A74C5EB8463B2DA51EE1B4 : {B456E582E9CA4FB19C4AF05AFDCCACB7={InputName=B456E582E9CA4FB19C4AF05AFDCCACB7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:29,217 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: B456E582E9CA4FB19C4AF05AFDCCACB7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:29,217 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4]
2016-01-14 18:31:29,217 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4]
2016-01-14 18:31:29,217 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4]
2016-01-14 18:31:29,218 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4]. Starting root input initializers: 1
2016-01-14 18:31:29,218 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:29,218 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850]
2016-01-14 18:31:29,217 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:29,218 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: B456E582E9CA4FB19C4AF05AFDCCACB7 on vertex vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4]
2016-01-14 18:31:29,220 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1452796289047_0001, dagName=[3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610] every/(1/1) ...latform/0_'field'_RESULTS
2016-01-14 18:31:29,221 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:29,221 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input B456E582E9CA4FB19C4AF05AFDCCACB7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:29,221 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:29,222 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:29,223 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:29,223 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:29,224 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850]
2016-01-14 18:31:29,224 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850]
2016-01-14 18:31:29,224 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850]
2016-01-14 18:31:29,224 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=AD3122A08F534D339DE80D1B49B33640, vertexId=vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:29,224 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=AD3122A08F534D339DE80D1B49B33640, vertexId=vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850]
2016-01-14 18:31:29,225 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: B456E582E9CA4FB19C4AF05AFDCCACB7 on vertex vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4]
2016-01-14 18:31:29,226 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=AD3122A08F534D339DE80D1B49B33640, vertexName=AA68ED39F0E0454291D3821BD0978850, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:29,226 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=AD3122A08F534D339DE80D1B49B33640, vertexId=vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850]
2016-01-14 18:31:29,227 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:29,227 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AA68ED39F0E0454291D3821BD0978850, vertexId=vertex_1452796289047_0001_1_01, initRequestedTime=1452796289218, initedTime=1452796289227, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:29,227 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:29,228 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:29,228 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {B456E582E9CA4FB19C4AF05AFDCCACB7=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:29,228 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4] parallelism set to 1
2016-01-14 18:31:29,228 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:29,229 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796289047_0001_1_00, eventCount=1
2016-01-14 18:31:29,229 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:29,230 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD775E21C6A74C5EB8463B2DA51EE1B4, vertexId=vertex_1452796289047_0001_1_00, initRequestedTime=1452796289217, initedTime=1452796289229, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:29,230 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4] with distanceFromRoot: 0
2016-01-14 18:31:29,230 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:29,230 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DD775E21C6A74C5EB8463B2DA51EE1B4
2016-01-14 18:31:29,230 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4]
2016-01-14 18:31:29,231 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:29,231 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796289047_0001_1_00, startRequestedTime=1452796289230, startedTime=1452796289230
2016-01-14 18:31:29,231 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:29,232 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:29,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:TASK_STARTED]: vertexName=DD775E21C6A74C5EB8463B2DA51EE1B4, taskId=task_1452796289047_0001_1_00_000000, scheduledTime=1452796289232, launchTime=1452796289232
2016-01-14 18:31:29,232 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289047_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:29,232 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796289047_0001_1_00 for vertex: vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850] numStartedSources: 1 numSources: 1
2016-01-14 18:31:29,233 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850] with distanceFromRoot: 1
2016-01-14 18:31:29,233 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4]
2016-01-14 18:31:29,233 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: DD775E21C6A74C5EB8463B2DA51EE1B4 in vertex: AA68ED39F0E0454291D3821BD0978850
2016-01-14 18:31:29,233 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: AA68ED39F0E0454291D3821BD0978850 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:29,234 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:29,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796289047_0001_1_01, startRequestedTime=1452796289233, startedTime=1452796289233
2016-01-14 18:31:29,234 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:29,234 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: DD775E21C6A74C5EB8463B2DA51EE1B4, VertexParallelism: 1, TaskAttemptID:attempt_1452796289047_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B456E582E9CA4FB19C4AF05AFDCCACB7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AA68ED39F0E0454291D3821BD0978850, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:29,235 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289047_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:29,235 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:29,236 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:29,236 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:29,236 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:29,237 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796289047_0001_00_000001
2016-01-14 18:31:29,237 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:29,238 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796289047_0001_00_000001
2016-01-14 18:31:29,238 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:29,239 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:29,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:29,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796289047_0001_00_000001, launchTime=1452796289239
2016-01-14 18:31:29,240 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796289047_0001_00_000001
2016-01-14 18:31:29,240 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796289047_0001_1_00_000000_0] to container: [container_1452796289047_0001_00_000001]
2016-01-14 18:31:29,240 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:29,241 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796289047_0001_1_00_000000_0] started. Is using containerId: [container_1452796289047_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:29,241 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD775E21C6A74C5EB8463B2DA51EE1B4, taskAttemptId=attempt_1452796289047_0001_1_00_000000_0, startTime=1452796289241, containerId=container_1452796289047_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796289047_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:29,241 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:29,241 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289047_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:29,241 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289047_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:29,241 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796289047_0001_00_000001 given task: attempt_1452796289047_0001_1_00_000000_0
2016-01-14 18:31:29,242 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796289047_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796289047_0001_1_00_000000_0
2016-01-14 18:31:29 Starting to run new task attempt: attempt_1452796289047_0001_1_00_000000_0
2016-01-14 18:31:29,242 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796289047_0001_1_00_000000_0
2016-01-14 18:31:29,242 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,242 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:29,242 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:29,243 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:29,243 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:29,243 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:29,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: DD775E21C6A74C5EB8463B2DA51EE1B4, VertexParallelism: 1, TaskAttemptID:attempt_1452796289047_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B456E582E9CA4FB19C4AF05AFDCCACB7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AA68ED39F0E0454291D3821BD0978850, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:29,243 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:29,243 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796289047_0001_1_00_000000_0
2016-01-14 18:31:29,243 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:29,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,244 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B456E582E9CA4FB19C4AF05AFDCCACB7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:29,245 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796289047_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:29,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:29,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B456E582E9CA4FB19C4AF05AFDCCACB7
2016-01-14 18:31:29,245 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,245 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: B456E582E9CA4FB19C4AF05AFDCCACB7
2016-01-14 18:31:29,247 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:29,247 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: B456E582E9CA4FB19C4AF05AFDCCACB7
2016-01-14 18:31:29,247 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B456E582E9CA4FB19C4AF05AFDCCACB7
2016-01-14 18:31:29,248 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796289047_0001, with dag id: dag_1452796289047_0001_1
2016-01-14 18:31:29,248 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:29,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AA68ED39F0E0454291D3821BD0978850, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:29,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:29,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AA68ED39F0E0454291D3821BD0978850
2016-01-14 18:31:29,252 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:29,252 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: AA68ED39F0E0454291D3821BD0978850
2016-01-14 18:31:29,252 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AA68ED39F0E0454291D3821BD0978850
2016-01-14 18:31:29,255 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD775E21C6A74C5EB8463B2DA51EE1B4, ordinal: 0
2016-01-14 18:31:29,256 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD775E21C6A74C5EB8463B2DA51EE1B4, mem on start (mb), free: 307, total: 368, max: 672
2016-01-14 18:31:29,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:29,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:29,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:29,256 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:29,256 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:29,256 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:29,256 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:29,256 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:29,257 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:29,258 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B456E582E9CA4FB19C4AF05AFDCCACB7, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:29,258 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AA68ED39F0E0454291D3821BD0978850, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:29,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:29,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B456E582E9CA4FB19C4AF05AFDCCACB7 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:29,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:29,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B456E582E9CA4FB19C4AF05AFDCCACB7
2016-01-14 18:31:29,259 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:29,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B456E582E9CA4FB19C4AF05AFDCCACB7
2016-01-14 18:31:29,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:29,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:29,260 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796289047_0001_1_00_000000_0
2016-01-14 18:31:29,260 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:29,261 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@43ea7d45
2016-01-14 18:31:29,262 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@43ea7d45
2016-01-14 18:31:29,262 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:29,262 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:29,264 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:29,265 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: B456E582E9CA4FB19C4AF05AFDCCACB7
2016-01-14 18:31:29,265 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: BC64DA8A809B4560BFB8195FEAE970C6
2016-01-14 18:31:29,265 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] BC64DA8A809B4560BFB8195FEAE970C6
2016-01-14 18:31:29,266 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1cea97ad; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6d8bedaf; comparator=cascading.tuple.hadoop.util.TupleComparator@7da1b458; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:29,266 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:29,266 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:29,267 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:29,280 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:29,280 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:29,281 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:29,281 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:29,281 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:29,282 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:29,282 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:29,282 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD775E21C6A74C5EB8463B2DA51EE1B4, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:29,285 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796289047_0001_1_00_000000_0
2016-01-14 18:31:29,285 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD775E21C6A74C5EB8463B2DA51EE1B4, mem on close (mb), free: 205, total: 368, max: 672
2016-01-14 18:31:29,286 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:29,286 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:29,286 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:29,286 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:29,288 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:29 Completed running task attempt: attempt_1452796289047_0001_1_00_000000_0
2016-01-14 18:31:29,291 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796289047_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:29,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796289283, Process_Duration=2, Process_End_Time=1452796289285, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:29,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796289047_0001_1_00, eventCount=1
2016-01-14 18:31:29,292 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:29,292 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,292 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796289047_0001_00_000001
2016-01-14 18:31:29,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD775E21C6A74C5EB8463B2DA51EE1B4, taskAttemptId=attempt_1452796289047_0001_1_00_000000_0, startTime=1452796289241, finishTime=1452796289292, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289283, Process_Duration=2, Process_End_Time=1452796289285, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:29,292 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289047_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:29,292 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:29,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:29,293 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796289047_0001_1_00_000000_0
2016-01-14 18:31:29,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:TASK_FINISHED]: vertexName=DD775E21C6A74C5EB8463B2DA51EE1B4, taskId=task_1452796289047_0001_1_00_000000, startTime=1452796289241, finishTime=1452796289293, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1452796289047_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289283, Process_Duration=2, Process_End_Time=1452796289285, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:29,294 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289047_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:29,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:29,294 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4] : 1
2016-01-14 18:31:29,294 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4]
2016-01-14 18:31:29,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:29,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:VERTEX_FINISHED]: vertexName=DD775E21C6A74C5EB8463B2DA51EE1B4, vertexId=vertex_1452796289047_0001_1_00, initRequestedTime=1452796289217, initedTime=1452796289229, startRequestedTime=1452796289230, startedTime=1452796289230, finishTime=1452796289294, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289283, Process_Duration=2, Process_End_Time=1452796289285, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796289241, firstTasksToStart=[ task_1452796289047_0001_1_00_000000 ], lastTaskFinishTime=1452796289292, lastTasksToFinish=[ task_1452796289047_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796289047_0001_1_00_000000 ], longestDurationTasks=[ task_1452796289047_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:29,295 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:29,295 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:29,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796289047_0001_00_000001, stoppedTime=1452796289295, exitStatus=0
2016-01-14 18:31:29,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:29,296 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:29,296 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850] attempt: attempt_1452796289047_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:29,296 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AA68ED39F0E0454291D3821BD0978850
2016-01-14 18:31:29,296 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850]
2016-01-14 18:31:29,296 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:29,296 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796289047_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:29,297 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:TASK_STARTED]: vertexName=AA68ED39F0E0454291D3821BD0978850, taskId=task_1452796289047_0001_1_01_000000, scheduledTime=1452796289297, launchTime=1452796289297
2016-01-14 18:31:29,297 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289047_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:29,297 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:29,297 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850]
2016-01-14 18:31:29,297 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:29,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: AA68ED39F0E0454291D3821BD0978850, VertexParallelism: 1, TaskAttemptID:attempt_1452796289047_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DD775E21C6A74C5EB8463B2DA51EE1B4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=AD3122A08F534D339DE80D1B49B33640, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:29,297 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289047_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:29,297 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:29,298 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796289047_0001_00_000002
2016-01-14 18:31:29,298 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:29,299 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796289047_0001_00_000002
2016-01-14 18:31:29,299 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:29,300 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:29,300 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796289047_0001_00_000002, launchTime=1452796289300
2016-01-14 18:31:29,301 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:29,301 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796289047_0001_00_000002
2016-01-14 18:31:29,302 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796289047_0001_1_01_000000_0] to container: [container_1452796289047_0001_00_000002]
2016-01-14 18:31:29,302 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796289047_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:29,303 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796289047_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:29,303 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796289047_0001_00_000001, Exiting
2016-01-14 18:31:29,303 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796289047_0001_00_000001
2016-01-14 18:31:29,303 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796289047_0001_00_000001
2016-01-14 18:31:29,303 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:29,304 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796289047_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:29,304 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796289047_0001_00_000001 completed successfully
2016-01-14 18:31:29,304 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:29,305 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796289047_0001_00_000002 given task: attempt_1452796289047_0001_1_01_000000_0
2016-01-14 18:31:29,305 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796289047_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796289047_0001_1_01_000000_0
2016-01-14 18:31:29,305 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796289047_0001_1_01_000000_0] started. Is using containerId: [container_1452796289047_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:29,305 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:29,305 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AA68ED39F0E0454291D3821BD0978850, taskAttemptId=attempt_1452796289047_0001_1_01_000000_0, startTime=1452796289305, containerId=container_1452796289047_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796289047_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:29,306 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289047_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:29 Starting to run new task attempt: attempt_1452796289047_0001_1_01_000000_0
2016-01-14 18:31:29,306 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796289047_0001_1_01_000000_0
2016-01-14 18:31:29,306 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,306 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289047_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:29,307 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:29,307 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:29,308 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:29,308 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:29,308 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:29,309 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610] every/(1/1) ...latform/0_'field'_RESULTS, VertexName: AA68ED39F0E0454291D3821BD0978850, VertexParallelism: 1, TaskAttemptID:attempt_1452796289047_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DD775E21C6A74C5EB8463B2DA51EE1B4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=AD3122A08F534D339DE80D1B49B33640, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:29,309 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:29,310 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796289047_0001_1_01_000000_0
2016-01-14 18:31:29,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:29,310 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,310 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796289047_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:29,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DD775E21C6A74C5EB8463B2DA51EE1B4, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:29,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:29,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AD3122A08F534D339DE80D1B49B33640, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:29,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DD775E21C6A74C5EB8463B2DA51EE1B4
2016-01-14 18:31:29,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:29,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AD3122A08F534D339DE80D1B49B33640
2016-01-14 18:31:29,313 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:29,313 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: AD3122A08F534D339DE80D1B49B33640
2016-01-14 18:31:29,314 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:29,314 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:29,314 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: DD775E21C6A74C5EB8463B2DA51EE1B4
2016-01-14 18:31:29,315 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DD775E21C6A74C5EB8463B2DA51EE1B4
2016-01-14 18:31:29,317 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:29,320 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:29,320 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AD3122A08F534D339DE80D1B49B33640
2016-01-14 18:31:29,323 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AA68ED39F0E0454291D3821BD0978850, ordinal: 1
2016-01-14 18:31:29,323 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AA68ED39F0E0454291D3821BD0978850, mem on start (mb), free: 199, total: 368, max: 672
2016-01-14 18:31:29,323 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:29,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:29,324 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:29,324 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:29,324 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:29,325 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:29,325 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:29,325 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:29,325 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:29,325 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AD3122A08F534D339DE80D1B49B33640, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:29,325 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DD775E21C6A74C5EB8463B2DA51EE1B4, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:29,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:29,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DD775E21C6A74C5EB8463B2DA51EE1B4
2016-01-14 18:31:29,326 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:29,326 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:29,327 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: DD775E21C6A74C5EB8463B2DA51EE1B4 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:29,327 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:29,327 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:29,327 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:29,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DD775E21C6A74C5EB8463B2DA51EE1B4
2016-01-14 18:31:29,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DD775E21C6A74C5EB8463B2DA51EE1B4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:29,329 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:29,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:29,330 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:29,330 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796289047_0001_1_01_000000_0
2016-01-14 18:31:29,332 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:29,332 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796289047_0001_1_00_000000_0_10129, runDuration: 39833, ]
2016-01-14 18:31:29,333 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: BC64DA8A809B4560BFB8195FEAE970C6
2016-01-14 18:31:29,333 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS"], id: AD3122A08F534D339DE80D1B49B33640
2016-01-14 18:31:29,333 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_RESULTS"]
2016-01-14 18:31:29,333 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] BC64DA8A809B4560BFB8195FEAE970C6, for 1 inputs
2016-01-14 18:31:29,334 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796289047_0001_1_00_000000_0_10129] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796289334, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:29,334 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : DD775E21C6A74C5EB8463B2DA51EE1B4
2016-01-14 18:31:29,334 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:29,334 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [DD775E21C6A74C5EB8463B2DA51EE1B4] #22 in 1ms
2016-01-14 18:31:29,334 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [DD775E21C6A74C5EB8463B2DA51EE1B4] #22, status:false, isInterrupted:false
2016-01-14 18:31:29,335 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:29,335 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:29,335 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:29,335 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:29,335 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:29,335 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : DD775E21C6A74C5EB8463B2DA51EE1B4
2016-01-14 18:31:29,336 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AA68ED39F0E0454291D3821BD0978850, all 1 inputs ready in: 00:00:00.003
2016-01-14 18:31:29,336 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:29,337 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5fed5095; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@55459766
2016-01-14 18:31:29,339 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:29,340 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:29,340 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796289047_0001_1_01_000000_0
2016-01-14 18:31:29,341 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796289047_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:29,342 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796289047_0001_1_01_000000_0
2016-01-14 18:31:29,342 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AA68ED39F0E0454291D3821BD0978850, mem on close (mb), free: 196, total: 368, max: 672
2016-01-14 18:31:29,342 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: DD775E21C6A74C5EB8463B2DA51EE1B4
2016-01-14 18:31:29,343 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:29,343 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:29,343 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:29,343 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:29,343 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:29 Completed running task attempt: attempt_1452796289047_0001_1_01_000000_0
2016-01-14 18:31:29,344 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796289047_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:29,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796289336, Process_Duration=6, Process_End_Time=1452796289342, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:29,345 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:29,345 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AA68ED39F0E0454291D3821BD0978850, taskAttemptId=attempt_1452796289047_0001_1_01_000000_0, startTime=1452796289305, finishTime=1452796289345, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289336, Process_Duration=6, Process_End_Time=1452796289342, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:29,345 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289047_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:29,345 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796289047_0001_1_01_000000_0
2016-01-14 18:31:29,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:29,345 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:29,346 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:29,346 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:TASK_FINISHED]: vertexName=AA68ED39F0E0454291D3821BD0978850, taskId=task_1452796289047_0001_1_01_000000, startTime=1452796289305, finishTime=1452796289345, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1452796289047_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289336, Process_Duration=6, Process_End_Time=1452796289342, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:29,346 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289047_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:29,346 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850] : 1
2016-01-14 18:31:29,346 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850]
2016-01-14 18:31:29,346 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796289047_0001_00_000002
2016-01-14 18:31:29,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:29,347 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:VERTEX_FINISHED]: vertexName=AA68ED39F0E0454291D3821BD0978850, vertexId=vertex_1452796289047_0001_1_01, initRequestedTime=1452796289218, initedTime=1452796289227, startRequestedTime=1452796289233, startedTime=1452796289233, finishTime=1452796289347, timeTaken=114, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386400256, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289336, Process_Duration=6, Process_End_Time=1452796289342, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796289305, firstTasksToStart=[ task_1452796289047_0001_1_01_000000 ], lastTaskFinishTime=1452796289345, lastTasksToFinish=[ task_1452796289047_0001_1_01_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796289047_0001_1_01_000000 ], longestDurationTasks=[ task_1452796289047_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:29,347 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:29,348 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:29,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796289047_0001_00_000002, stoppedTime=1452796289348, exitStatus=0
2016-01-14 18:31:29,348 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:29,348 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:29,348 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:29,348 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796289047_0001_1
2016-01-14 18:31:29,348 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796289047_0001_1
2016-01-14 18:31:29,349 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796289047_0001_1_00 [DD775E21C6A74C5EB8463B2DA51EE1B4]
2016-01-14 18:31:29,349 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: AD3122A08F534D339DE80D1B49B33640 for vertex: vertex_1452796289047_0001_1_01 [AA68ED39F0E0454291D3821BD0978850]
2016-01-14 18:31:29,357 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796289047_0001_1, queueSize=0
2016-01-14 18:31:29,357 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796289047_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:29,357 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:29,357 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796289047_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:29,357 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796289047_0001_00_000002, Exiting
2016-01-14 18:31:29,357 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796289047_0001_00_000002
2016-01-14 18:31:29,357 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796289047_0001_00_000002
2016-01-14 18:31:29,357 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796289047_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:29,358 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796289047_0001_00_000002 completed successfully
2016-01-14 18:31:29,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289047_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796289047_0001_1, startTime=1452796289217, finishTime=1452796289355, timeTaken=138, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=772800512, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592578619, Process_Duration=8, Process_End_Time=2905592578627, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:29,358 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796289047_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:29,358 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796289047_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:29,358 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289047_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:29,358 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796289047_0001_00_000002
2016-01-14 18:31:29 Completed Dag: dag_1452796289047_0001_1
2016-01-14 18:31:29,360 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796289047_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:29,360 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796289047_0001_1_post
2016-01-14 18:31:29,360 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,360 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:29,364 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_'field'_RESULTS, applicationId=application_1452796289047_0001
2016-01-14 18:31:29,366 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:29,366 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:29,366 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:29,366 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:29,366 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:29,367 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:29,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:29,367 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:29,367 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:29,367 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:29,367 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:29,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:29,368 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:29,372 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/3AD5235265374900AE7449D224603F7F/AE4B7107DE6E4EB6B65F4CDB96312610/.tez/application_1452796289047_0001
2016-01-14 18:31:29,372 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.344
2016-01-14 18:31:29,398 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:29,398 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:29,456 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_RESULTS
2016-01-14 18:31:29,480 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:29,480 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:29,515 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 18:31:29,521 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 18:31:29,521 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:29,521 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:29,522 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:29,537 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:29,543 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL
2016-01-14 18:31:29,590 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:29,590 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:29,591 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL"]
2016-01-14 18:31:29,591 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:29,591 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:29,591 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:29,592 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/0_'field'_ALL
2016-01-14 18:31:29,592 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:29,605 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:29,609 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:29,610 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:29,610 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:29,620 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0/.tez/application_1452796289609_0001 doesn't exist and is created
2016-01-14 18:31:29,636 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:29,636 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:29,636 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0/.tez/application_1452796289609_0001
2016-01-14 18:31:29,638 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796289609_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:29,649 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:29,650 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:29,651 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:29,651 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:29,651 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0/.tez/application_1452796289609_0001/localmode-log-dir
2016-01-14 18:31:29,651 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0/.tez/application_1452796289609_0001/localmode-log-dir/history.txt.appattempt_1452796289609_0001_000000, maxErrors=10
2016-01-14 18:31:29,652 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:29,652 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796289609_0001_000000, appSubmitTime=1452796289638, launchTime=1452796289638
2016-01-14 18:31:29,654 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:29,655 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:29,656 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:29,656 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:29,657 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:46817
2016-01-14 18:31:29,659 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:29,660 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796289609_0001_000000, startTime=1452796289660
2016-01-14 18:31:29,660 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:29,661 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:29,736 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:29,736 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:29,736 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1452796289609_0001, dagName=[61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0] every/(1/1) ...blyplatform/0_'field'_ALL
2016-01-14 18:31:29,763 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0] every/(1/1) ...blyplatform/0_'field'_ALL
2016-01-14 18:31:29,764 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796289609_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0/.tez/application_1452796289609_0001/localmode-log-dir/dag_1452796289609_0001_1.dot
2016-01-14 18:31:29,764 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0/.tez/application_1452796289609_0001/localmode-log-dir/dag_1452796289609_0001_1-tez-dag.pb.txt
2016-01-14 18:31:29,770 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796289609_0001_1
2016-01-14 18:31:29,770 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,771 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0] every/(1/1) ...blyplatform/0_'field'_ALL
2016-01-14 18:31:29 Running Dag: dag_1452796289609_0001_1
2016-01-14 18:31:29,780 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796289609_0001_1, submitTime=1452796289763
2016-01-14 18:31:29,780 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:29,781 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FE59EEBF5BF64401961310084477B6CB
2016-01-14 18:31:29,781 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:29,782 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796289609_0001_1, initTime=1452796289780
2016-01-14 18:31:29,782 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:29,782 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796289609_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:29,783 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:DAG_STARTED]: dagID=dag_1452796289609_0001_1, startTime=1452796289783
2016-01-14 18:31:29,783 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:29,783 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:29,783 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796289609_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:29,783 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: DC0E27BBA2BA41F9A28A274CF1E9E47D : {33320B6F86064011AA90A246152AD453={InputName=33320B6F86064011AA90A246152AD453}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:29,783 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 33320B6F86064011AA90A246152AD453, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:29,783 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1452796289609_0001, dagName=[61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0] every/(1/1) ...blyplatform/0_'field'_ALL
2016-01-14 18:31:29,784 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D]
2016-01-14 18:31:29,784 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D]
2016-01-14 18:31:29,784 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D]
2016-01-14 18:31:29,784 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D]. Starting root input initializers: 1
2016-01-14 18:31:29,785 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:29,785 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 33320B6F86064011AA90A246152AD453 on vertex vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D]
2016-01-14 18:31:29,785 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB]
2016-01-14 18:31:29,786 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:29,787 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 33320B6F86064011AA90A246152AD453 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:29,787 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:29,787 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:29,788 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:29,788 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:29,789 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB]
2016-01-14 18:31:29,789 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 33320B6F86064011AA90A246152AD453 on vertex vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D]
2016-01-14 18:31:29,789 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB]
2016-01-14 18:31:29,789 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB]
2016-01-14 18:31:29,789 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=212924E82F3D4BFCB1A0533F225D81D7, vertexId=vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:29,789 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=212924E82F3D4BFCB1A0533F225D81D7, vertexId=vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB]
2016-01-14 18:31:29,791 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=212924E82F3D4BFCB1A0533F225D81D7, vertexName=FE59EEBF5BF64401961310084477B6CB, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:29,791 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=212924E82F3D4BFCB1A0533F225D81D7, vertexId=vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB]
2016-01-14 18:31:29,792 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:29,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FE59EEBF5BF64401961310084477B6CB, vertexId=vertex_1452796289609_0001_1_01, initRequestedTime=1452796289785, initedTime=1452796289792, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:29,793 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:29,793 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:29,793 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {33320B6F86064011AA90A246152AD453=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:29,793 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D] parallelism set to 1
2016-01-14 18:31:29,793 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:29,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796289609_0001_1_00, eventCount=1
2016-01-14 18:31:29,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DC0E27BBA2BA41F9A28A274CF1E9E47D, vertexId=vertex_1452796289609_0001_1_00, initRequestedTime=1452796289783, initedTime=1452796289794, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:29,794 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:29,795 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D] with distanceFromRoot: 0
2016-01-14 18:31:29,795 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:29,795 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in DC0E27BBA2BA41F9A28A274CF1E9E47D
2016-01-14 18:31:29,795 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D]
2016-01-14 18:31:29,796 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:29,796 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796289609_0001_1_00, startRequestedTime=1452796289795, startedTime=1452796289795
2016-01-14 18:31:29,796 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:29,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:29,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:TASK_STARTED]: vertexName=DC0E27BBA2BA41F9A28A274CF1E9E47D, taskId=task_1452796289609_0001_1_00_000000, scheduledTime=1452796289797, launchTime=1452796289797
2016-01-14 18:31:29,797 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289609_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:29,797 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796289609_0001_1_00 for vertex: vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB] numStartedSources: 1 numSources: 1
2016-01-14 18:31:29,797 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB] with distanceFromRoot: 1
2016-01-14 18:31:29,798 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D]
2016-01-14 18:31:29,798 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: DC0E27BBA2BA41F9A28A274CF1E9E47D in vertex: FE59EEBF5BF64401961310084477B6CB
2016-01-14 18:31:29,798 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FE59EEBF5BF64401961310084477B6CB with 1 source tasks and 1 pending tasks
2016-01-14 18:31:29,798 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:29,799 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796289609_0001_1_01, startRequestedTime=1452796289798, startedTime=1452796289798
2016-01-14 18:31:29,799 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:29,799 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: DC0E27BBA2BA41F9A28A274CF1E9E47D, VertexParallelism: 1, TaskAttemptID:attempt_1452796289609_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=33320B6F86064011AA90A246152AD453, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FE59EEBF5BF64401961310084477B6CB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:29,799 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289609_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:29,799 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:29,800 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:29,800 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:29,800 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:29,801 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796289609_0001_00_000001
2016-01-14 18:31:29,801 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:29,802 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796289609_0001_00_000001
2016-01-14 18:31:29,802 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:29,803 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:29,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:29,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796289609_0001_00_000001, launchTime=1452796289803
2016-01-14 18:31:29,806 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796289609_0001_00_000001
2016-01-14 18:31:29,806 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796289609_0001_1_00_000000_0] to container: [container_1452796289609_0001_00_000001]
2016-01-14 18:31:29,806 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:29,807 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796289609_0001_1_00_000000_0] started. Is using containerId: [container_1452796289609_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:29,807 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DC0E27BBA2BA41F9A28A274CF1E9E47D, taskAttemptId=attempt_1452796289609_0001_1_00_000000_0, startTime=1452796289807, containerId=container_1452796289609_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796289609_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:29,807 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:29,807 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289609_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:29,808 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289609_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:29,808 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796289609_0001_00_000001 given task: attempt_1452796289609_0001_1_00_000000_0
2016-01-14 18:31:29,808 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796289609_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796289609_0001_1_00_000000_0
2016-01-14 18:31:29 Starting to run new task attempt: attempt_1452796289609_0001_1_00_000000_0
2016-01-14 18:31:29,809 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796289609_0001_1_00_000000_0
2016-01-14 18:31:29,809 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,809 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:29,809 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:29,809 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:29,810 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:29,810 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:29,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: DC0E27BBA2BA41F9A28A274CF1E9E47D, VertexParallelism: 1, TaskAttemptID:attempt_1452796289609_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=33320B6F86064011AA90A246152AD453, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FE59EEBF5BF64401961310084477B6CB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:29,810 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:29,811 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796289609_0001_1_00_000000_0
2016-01-14 18:31:29,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:29,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,812 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796289609_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:29,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=33320B6F86064011AA90A246152AD453, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:29,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:29,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 33320B6F86064011AA90A246152AD453
2016-01-14 18:31:29,813 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 33320B6F86064011AA90A246152AD453
2016-01-14 18:31:29,814 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796289609_0001, with dag id: dag_1452796289609_0001_1
2016-01-14 18:31:29,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FE59EEBF5BF64401961310084477B6CB, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:29,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:29,816 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FE59EEBF5BF64401961310084477B6CB
2016-01-14 18:31:29,818 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:29,818 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:29,819 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FE59EEBF5BF64401961310084477B6CB
2016-01-14 18:31:29,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FE59EEBF5BF64401961310084477B6CB
2016-01-14 18:31:29,821 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:29,822 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 33320B6F86064011AA90A246152AD453
2016-01-14 18:31:29,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 33320B6F86064011AA90A246152AD453
2016-01-14 18:31:29,826 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DC0E27BBA2BA41F9A28A274CF1E9E47D, ordinal: 0
2016-01-14 18:31:29,827 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DC0E27BBA2BA41F9A28A274CF1E9E47D, mem on start (mb), free: 304, total: 369, max: 672
2016-01-14 18:31:29,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:29,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:29,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:29,828 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:29,829 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:29,829 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:29,830 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:29,830 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:29,830 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:29,831 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 33320B6F86064011AA90A246152AD453, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:29,831 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FE59EEBF5BF64401961310084477B6CB, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:29,831 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:29,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 33320B6F86064011AA90A246152AD453
2016-01-14 18:31:29,832 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:29,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 33320B6F86064011AA90A246152AD453 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:29,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:29,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:29,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 33320B6F86064011AA90A246152AD453
2016-01-14 18:31:29,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:29,836 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796289609_0001_1_00_000000_0
2016-01-14 18:31:29,836 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:29,837 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2ea2a4e4
2016-01-14 18:31:29,838 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2ea2a4e4
2016-01-14 18:31:29,838 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:29,838 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:29,839 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:29,839 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 33320B6F86064011AA90A246152AD453
2016-01-14 18:31:29,840 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 6FB4597DC20D438387AC9784E0D3F1A3
2016-01-14 18:31:29,840 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 6FB4597DC20D438387AC9784E0D3F1A3
2016-01-14 18:31:29,840 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@34c857a0; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@4cf0e472; comparator=cascading.tuple.hadoop.util.TupleComparator@5b261fd4; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:29,841 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:29,841 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:29,841 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:29,854 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:29,855 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:29,855 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:29,855 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:29,855 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:29,855 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:29,855 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:29,856 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DC0E27BBA2BA41F9A28A274CF1E9E47D, all 1 inputs ready in: 00:00:00.001
2016-01-14 18:31:29,859 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796289609_0001_1_00_000000_0
2016-01-14 18:31:29,859 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DC0E27BBA2BA41F9A28A274CF1E9E47D, mem on close (mb), free: 202, total: 369, max: 672
2016-01-14 18:31:29,859 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:29,859 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:29,859 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:29,859 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:29,861 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:29 Completed running task attempt: attempt_1452796289609_0001_1_00_000000_0
2016-01-14 18:31:29,863 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796289609_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:29,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796289609_0001_1_00, eventCount=1
2016-01-14 18:31:29,865 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796289856, Process_Duration=3, Process_End_Time=1452796289859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:29,866 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DC0E27BBA2BA41F9A28A274CF1E9E47D, taskAttemptId=attempt_1452796289609_0001_1_00_000000_0, startTime=1452796289807, finishTime=1452796289864, timeTaken=57, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289856, Process_Duration=3, Process_End_Time=1452796289859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:29,866 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:29,866 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289609_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:29,866 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:29,866 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:29,867 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,866 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796289609_0001_1_00_000000_0
2016-01-14 18:31:29,867 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796289609_0001_00_000001
2016-01-14 18:31:29,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:29,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:TASK_FINISHED]: vertexName=DC0E27BBA2BA41F9A28A274CF1E9E47D, taskId=task_1452796289609_0001_1_00_000000, startTime=1452796289807, finishTime=1452796289867, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1452796289609_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289856, Process_Duration=3, Process_End_Time=1452796289859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:29,868 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289609_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:29,868 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D] : 1
2016-01-14 18:31:29,868 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D]
2016-01-14 18:31:29,868 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:29,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:VERTEX_FINISHED]: vertexName=DC0E27BBA2BA41F9A28A274CF1E9E47D, vertexId=vertex_1452796289609_0001_1_00, initRequestedTime=1452796289783, initedTime=1452796289794, startRequestedTime=1452796289795, startedTime=1452796289795, finishTime=1452796289868, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289856, Process_Duration=3, Process_End_Time=1452796289859, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796289807, firstTasksToStart=[ task_1452796289609_0001_1_00_000000 ], lastTaskFinishTime=1452796289864, lastTasksToFinish=[ task_1452796289609_0001_1_00_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796289609_0001_1_00_000000 ], longestDurationTasks=[ task_1452796289609_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:29,869 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:29,869 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:29,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796289609_0001_00_000001, stoppedTime=1452796289869, exitStatus=0
2016-01-14 18:31:29,869 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:29,869 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:29,869 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB] attempt: attempt_1452796289609_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:29,870 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FE59EEBF5BF64401961310084477B6CB
2016-01-14 18:31:29,870 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB]
2016-01-14 18:31:29,870 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:29,870 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796289609_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:29,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:TASK_STARTED]: vertexName=FE59EEBF5BF64401961310084477B6CB, taskId=task_1452796289609_0001_1_01_000000, scheduledTime=1452796289870, launchTime=1452796289870
2016-01-14 18:31:29,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:29,871 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289609_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:29,871 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:29,871 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB]
2016-01-14 18:31:29,871 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: FE59EEBF5BF64401961310084477B6CB, VertexParallelism: 1, TaskAttemptID:attempt_1452796289609_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DC0E27BBA2BA41F9A28A274CF1E9E47D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=212924E82F3D4BFCB1A0533F225D81D7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:29,872 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289609_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:29,872 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:29,872 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796289609_0001_00_000002
2016-01-14 18:31:29,872 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:29,874 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796289609_0001_00_000002
2016-01-14 18:31:29,874 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:29,875 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:29,875 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:29,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796289609_0001_00_000002, launchTime=1452796289875
2016-01-14 18:31:29,876 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796289609_0001_00_000002
2016-01-14 18:31:29,877 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796289609_0001_1_01_000000_0] to container: [container_1452796289609_0001_00_000002]
2016-01-14 18:31:29,877 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:29,878 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796289609_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:29,878 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796289609_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:29,878 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796289609_0001_00_000001, Exiting
2016-01-14 18:31:29,878 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796289609_0001_00_000001
2016-01-14 18:31:29,878 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796289609_0001_00_000001
2016-01-14 18:31:29,879 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796289609_0001_00_000002 given task: attempt_1452796289609_0001_1_01_000000_0
2016-01-14 18:31:29,879 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796289609_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796289609_0001_1_01_000000_0
2016-01-14 18:31:29,879 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796289609_0001_1_01_000000_0] started. Is using containerId: [container_1452796289609_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:29,880 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796289609_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:29,880 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796289609_0001_00_000001 completed successfully
2016-01-14 18:31:29 Starting to run new task attempt: attempt_1452796289609_0001_1_01_000000_0
2016-01-14 18:31:29,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FE59EEBF5BF64401961310084477B6CB, taskAttemptId=attempt_1452796289609_0001_1_01_000000_0, startTime=1452796289879, containerId=container_1452796289609_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796289609_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:29,880 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:29,880 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796289609_0001_1_01_000000_0
2016-01-14 18:31:29,881 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,881 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:29,881 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:29,881 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:29,881 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:29,881 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:29,881 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0] every/(1/1) ...blyplatform/0_'field'_ALL, VertexName: FE59EEBF5BF64401961310084477B6CB, VertexParallelism: 1, TaskAttemptID:attempt_1452796289609_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=DC0E27BBA2BA41F9A28A274CF1E9E47D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=212924E82F3D4BFCB1A0533F225D81D7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:29,882 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:29,881 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289609_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:29,882 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:29,882 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289609_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:29,883 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796289609_0001_1_01_000000_0
2016-01-14 18:31:29,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:29,883 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,884 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=DC0E27BBA2BA41F9A28A274CF1E9E47D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:29,884 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:29,884 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: DC0E27BBA2BA41F9A28A274CF1E9E47D
2016-01-14 18:31:29,884 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796289609_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:29,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,886 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=212924E82F3D4BFCB1A0533F225D81D7, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:29,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:29,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 212924E82F3D4BFCB1A0533F225D81D7
2016-01-14 18:31:29,887 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:29,887 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 212924E82F3D4BFCB1A0533F225D81D7
2016-01-14 18:31:29,889 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:29,897 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:29,897 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 212924E82F3D4BFCB1A0533F225D81D7
2016-01-14 18:31:29,898 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FE59EEBF5BF64401961310084477B6CB, ordinal: 1
2016-01-14 18:31:29,898 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE59EEBF5BF64401961310084477B6CB, mem on start (mb), free: 197, total: 369, max: 672
2016-01-14 18:31:29,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:29,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:29,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:29,899 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:29,900 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:29,900 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: DC0E27BBA2BA41F9A28A274CF1E9E47D
2016-01-14 18:31:29,900 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: DC0E27BBA2BA41F9A28A274CF1E9E47D
2016-01-14 18:31:29,901 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:29,901 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:29,901 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:29,902 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:29,902 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:29,903 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:29,903 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:29,904 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 212924E82F3D4BFCB1A0533F225D81D7, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:29,904 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, DC0E27BBA2BA41F9A28A274CF1E9E47D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:29,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:29,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: DC0E27BBA2BA41F9A28A274CF1E9E47D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:29,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: DC0E27BBA2BA41F9A28A274CF1E9E47D
2016-01-14 18:31:29,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:29,906 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:29,906 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:29,907 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:29,907 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: DC0E27BBA2BA41F9A28A274CF1E9E47D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:29,907 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:29,907 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:29,907 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:29,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: DC0E27BBA2BA41F9A28A274CF1E9E47D
2016-01-14 18:31:29,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:29,909 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796289609_0001_1_01_000000_0
2016-01-14 18:31:29,909 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796289609_0001_1_00_000000_0_10135, runDuration: 45931, ]
2016-01-14 18:31:29,910 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796289609_0001_1_00_000000_0_10135] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796289910, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:29,910 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : DC0E27BBA2BA41F9A28A274CF1E9E47D
2016-01-14 18:31:29,910 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:29,910 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [DC0E27BBA2BA41F9A28A274CF1E9E47D] #23 in 1ms
2016-01-14 18:31:29,911 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [DC0E27BBA2BA41F9A28A274CF1E9E47D] #23, status:false, isInterrupted:false
2016-01-14 18:31:29,911 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:29,911 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:29,911 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:29,912 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:29,912 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:29,912 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : DC0E27BBA2BA41F9A28A274CF1E9E47D
2016-01-14 18:31:29,912 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:29,913 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:29,914 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 6FB4597DC20D438387AC9784E0D3F1A3
2016-01-14 18:31:29,914 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL"], id: 212924E82F3D4BFCB1A0533F225D81D7
2016-01-14 18:31:29,914 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_ALL"]
2016-01-14 18:31:29,915 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 6FB4597DC20D438387AC9784E0D3F1A3, for 1 inputs
2016-01-14 18:31:29,915 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FE59EEBF5BF64401961310084477B6CB, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:29,915 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5a10836b; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@722413fe
2016-01-14 18:31:29,920 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:29,920 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:29,921 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796289609_0001_1_01_000000_0
2016-01-14 18:31:29,921 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796289609_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:29,921 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796289609_0001_1_01_000000_0
2016-01-14 18:31:29,921 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FE59EEBF5BF64401961310084477B6CB, mem on close (mb), free: 194, total: 369, max: 672
2016-01-14 18:31:29,921 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: DC0E27BBA2BA41F9A28A274CF1E9E47D
2016-01-14 18:31:29,922 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:29,922 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:29,922 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:29,922 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:29,922 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:29 Completed running task attempt: attempt_1452796289609_0001_1_01_000000_0
2016-01-14 18:31:29,923 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796289609_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:29,923 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796289915, Process_Duration=6, Process_End_Time=1452796289921, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:29,924 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:29,924 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:29,924 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FE59EEBF5BF64401961310084477B6CB, taskAttemptId=attempt_1452796289609_0001_1_01_000000_0, startTime=1452796289879, finishTime=1452796289923, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289915, Process_Duration=6, Process_End_Time=1452796289921, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:29,924 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796289609_0001_00_000002
2016-01-14 18:31:29,924 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796289609_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:29,924 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796289609_0001_1_01_000000_0
2016-01-14 18:31:29,924 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:29,925 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:29,925 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:TASK_FINISHED]: vertexName=FE59EEBF5BF64401961310084477B6CB, taskId=task_1452796289609_0001_1_01_000000, startTime=1452796289879, finishTime=1452796289924, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452796289609_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289915, Process_Duration=6, Process_End_Time=1452796289921, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:29,925 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796289609_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:29,925 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB] : 1
2016-01-14 18:31:29,925 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB]
2016-01-14 18:31:29,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:29,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:VERTEX_FINISHED]: vertexName=FE59EEBF5BF64401961310084477B6CB, vertexId=vertex_1452796289609_0001_1_01, initRequestedTime=1452796289785, initedTime=1452796289792, startRequestedTime=1452796289798, startedTime=1452796289798, finishTime=1452796289925, timeTaken=127, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=386924544, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796289915, Process_Duration=6, Process_End_Time=1452796289921, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796289879, firstTasksToStart=[ task_1452796289609_0001_1_01_000000 ], lastTaskFinishTime=1452796289923, lastTasksToFinish=[ task_1452796289609_0001_1_01_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796289609_0001_1_01_000000 ], longestDurationTasks=[ task_1452796289609_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:29,926 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:29,927 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:29,927 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796289609_0001_00_000002, stoppedTime=1452796289927, exitStatus=0
2016-01-14 18:31:29,927 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:29,927 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:29,927 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:29,928 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796289609_0001_1
2016-01-14 18:31:29,928 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796289609_0001_1
2016-01-14 18:31:29,929 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 212924E82F3D4BFCB1A0533F225D81D7 for vertex: vertex_1452796289609_0001_1_01 [FE59EEBF5BF64401961310084477B6CB]
2016-01-14 18:31:29,934 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796289609_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:29,934 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796289609_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:29,935 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796289609_0001_00_000002, Exiting
2016-01-14 18:31:29,935 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796289609_0001_00_000002
2016-01-14 18:31:29,935 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796289609_0001_00_000002
2016-01-14 18:31:29,935 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796289609_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:29,935 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796289609_0001_00_000002 completed successfully
2016-01-14 18:31:29,936 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796289609_0001_1_00 [DC0E27BBA2BA41F9A28A274CF1E9E47D]
2016-01-14 18:31:29,938 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796289609_0001_1, queueSize=0
2016-01-14 18:31:29,938 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:29,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796289609_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796289609_0001_1, startTime=1452796289783, finishTime=1452796289936, timeTaken=153, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=571, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=773849088, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592579771, Process_Duration=9, Process_End_Time=2905592579780, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:29,939 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796289609_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:29,939 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796289609_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:29,939 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796289609_0001_00_000002
2016-01-14 18:31:29,939 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796289609_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:29 Completed Dag: dag_1452796289609_0001_1
2016-01-14 18:31:29,945 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796289609_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:29,945 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796289609_0001_1_post
2016-01-14 18:31:29,945 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:29,945 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:29,947 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_'field'_ALL, applicationId=application_1452796289609_0001
2016-01-14 18:31:29,950 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:29,950 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:29,950 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:29,950 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:29,950 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:29,951 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:29,951 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:29,951 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:29,951 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:29,951 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:29,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:29,952 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:29,953 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:29,957 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/61F39F6416AE4D9F954C0E82536B17B7/AB53776192C1411B96EE888C7315CBF0/.tez/application_1452796289609_0001
2016-01-14 18:31:29,958 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.367
2016-01-14 18:31:29,960 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:29,960 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:30,029 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_ALL
2016-01-14 18:31:30,053 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:30,053 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:30,081 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-01-14 18:31:30,081 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-01-14 18:31:30,082 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:30,082 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:30,082 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_REPLACE
2016-01-14 18:31:30,107 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:30,107 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:30,147 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 18:31:30,149 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 18:31:30,150 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:30,150 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:30,150 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:30,167 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:30,173 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP
2016-01-14 18:31:30,234 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:30,234 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:30,234 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP"]
2016-01-14 18:31:30,234 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:30,234 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:30,234 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:30,235 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/0_'field'_SWAP
2016-01-14 18:31:30,235 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:30,248 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:30,253 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:30,253 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:30,253 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:30,263 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8/.tez/application_1452796290253_0001 doesn't exist and is created
2016-01-14 18:31:30,279 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:30,279 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:30,280 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8/.tez/application_1452796290253_0001
2016-01-14 18:31:30,282 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796290253_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:30,300 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:30,301 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:30,303 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:30,303 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:30,303 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8/.tez/application_1452796290253_0001/localmode-log-dir
2016-01-14 18:31:30,304 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8/.tez/application_1452796290253_0001/localmode-log-dir/history.txt.appattempt_1452796290253_0001_000000, maxErrors=10
2016-01-14 18:31:30,304 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:30,304 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796290253_0001_000000, appSubmitTime=1452796290282, launchTime=1452796290282
2016-01-14 18:31:30,308 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:30,311 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:30,312 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:30,312 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:30,314 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:55298
2016-01-14 18:31:30,317 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:30,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796290253_0001_000000, startTime=1452796290319
2016-01-14 18:31:30,319 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:30,319 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:30,379 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:30,380 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:30,380 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1452796290253_0001, dagName=[609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-01-14 18:31:30,396 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-01-14 18:31:30,398 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796290253_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8/.tez/application_1452796290253_0001/localmode-log-dir/dag_1452796290253_0001_1.dot
2016-01-14 18:31:30,398 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8/.tez/application_1452796290253_0001/localmode-log-dir/dag_1452796290253_0001_1-tez-dag.pb.txt
2016-01-14 18:31:30,404 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796290253_0001_1
2016-01-14 18:31:30,405 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:30,405 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-01-14 18:31:30 Running Dag: dag_1452796290253_0001_1
2016-01-14 18:31:30,414 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796290253_0001_1, submitTime=1452796290397
2016-01-14 18:31:30,414 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:30,415 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F17459C26ACC4A2783B8F069A238FF18
2016-01-14 18:31:30,416 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:30,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796290253_0001_1, initTime=1452796290415
2016-01-14 18:31:30,416 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796290253_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:30,416 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:30,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:DAG_STARTED]: dagID=dag_1452796290253_0001_1, startTime=1452796290416
2016-01-14 18:31:30,417 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:30,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:30,417 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1452796290253_0001, dagName=[609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8] every/(1/1) ...lyplatform/0_'field'_SWAP
2016-01-14 18:31:30,417 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796290253_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:30,418 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A74D81EEBF854B3399989ED07D49E0DA : {E5010A52968349B2861DEA585CB100ED={InputName=E5010A52968349B2861DEA585CB100ED}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:30,418 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E5010A52968349B2861DEA585CB100ED, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:30,418 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA]
2016-01-14 18:31:30,418 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA]
2016-01-14 18:31:30,419 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA]
2016-01-14 18:31:30,419 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA]. Starting root input initializers: 1
2016-01-14 18:31:30,419 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:30,419 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18]
2016-01-14 18:31:30,422 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:30,422 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18]
2016-01-14 18:31:30,422 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18]
2016-01-14 18:31:30,422 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18]
2016-01-14 18:31:30,422 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C59DFDC26FDB456597B918E5F869E910, vertexId=vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:30,423 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C59DFDC26FDB456597B918E5F869E910, vertexId=vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18]
2016-01-14 18:31:30,424 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C59DFDC26FDB456597B918E5F869E910, vertexName=F17459C26ACC4A2783B8F069A238FF18, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:30,424 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C59DFDC26FDB456597B918E5F869E910, vertexId=vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18]
2016-01-14 18:31:30,425 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:30,425 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E5010A52968349B2861DEA585CB100ED on vertex vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA]
2016-01-14 18:31:30,425 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F17459C26ACC4A2783B8F069A238FF18, vertexId=vertex_1452796290253_0001_1_01, initRequestedTime=1452796290419, initedTime=1452796290425, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:30,427 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:30,427 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:30,427 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E5010A52968349B2861DEA585CB100ED asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:30,427 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:30,428 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:30,430 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:30,430 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E5010A52968349B2861DEA585CB100ED on vertex vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA]
2016-01-14 18:31:30,431 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:30,431 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E5010A52968349B2861DEA585CB100ED=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:30,431 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA] parallelism set to 1
2016-01-14 18:31:30,432 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:30,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796290253_0001_1_00, eventCount=1
2016-01-14 18:31:30,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:30,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A74D81EEBF854B3399989ED07D49E0DA, vertexId=vertex_1452796290253_0001_1_00, initRequestedTime=1452796290418, initedTime=1452796290432, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:30,433 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA] with distanceFromRoot: 0
2016-01-14 18:31:30,433 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:30,433 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A74D81EEBF854B3399989ED07D49E0DA
2016-01-14 18:31:30,433 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA]
2016-01-14 18:31:30,434 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:30,434 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796290253_0001_1_00, startRequestedTime=1452796290433, startedTime=1452796290433
2016-01-14 18:31:30,434 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796290253_0001, with dag id: dag_1452796290253_0001_1
2016-01-14 18:31:30,434 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:30,435 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:30,435 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:TASK_STARTED]: vertexName=A74D81EEBF854B3399989ED07D49E0DA, taskId=task_1452796290253_0001_1_00_000000, scheduledTime=1452796290435, launchTime=1452796290435
2016-01-14 18:31:30,435 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290253_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:30,435 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796290253_0001_1_00 for vertex: vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18] numStartedSources: 1 numSources: 1
2016-01-14 18:31:30,436 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18] with distanceFromRoot: 1
2016-01-14 18:31:30,437 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA]
2016-01-14 18:31:30,437 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A74D81EEBF854B3399989ED07D49E0DA in vertex: F17459C26ACC4A2783B8F069A238FF18
2016-01-14 18:31:30,438 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F17459C26ACC4A2783B8F069A238FF18 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:30,438 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:30,438 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796290253_0001_1_01, startRequestedTime=1452796290437, startedTime=1452796290437
2016-01-14 18:31:30,438 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:30,439 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: A74D81EEBF854B3399989ED07D49E0DA, VertexParallelism: 1, TaskAttemptID:attempt_1452796290253_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E5010A52968349B2861DEA585CB100ED, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F17459C26ACC4A2783B8F069A238FF18, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:30,439 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290253_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:30,440 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:30,440 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:30,441 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:30,441 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:30,441 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796290253_0001_00_000001
2016-01-14 18:31:30,441 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:30,443 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796290253_0001_00_000001
2016-01-14 18:31:30,443 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:30,443 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:30,444 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:30,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796290253_0001_00_000001, launchTime=1452796290443
2016-01-14 18:31:30,444 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796290253_0001_00_000001
2016-01-14 18:31:30,445 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796290253_0001_1_00_000000_0] to container: [container_1452796290253_0001_00_000001]
2016-01-14 18:31:30,445 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:30,445 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796290253_0001_1_00_000000_0] started. Is using containerId: [container_1452796290253_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:30,446 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:30,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A74D81EEBF854B3399989ED07D49E0DA, taskAttemptId=attempt_1452796290253_0001_1_00_000000_0, startTime=1452796290445, containerId=container_1452796290253_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796290253_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:30,446 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290253_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:30,446 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290253_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:30,446 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796290253_0001_00_000001 given task: attempt_1452796290253_0001_1_00_000000_0
2016-01-14 18:31:30,447 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796290253_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796290253_0001_1_00_000000_0
2016-01-14 18:31:30 Starting to run new task attempt: attempt_1452796290253_0001_1_00_000000_0
2016-01-14 18:31:30,447 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796290253_0001_1_00_000000_0
2016-01-14 18:31:30,447 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:30,447 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:30,447 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:30,447 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:30,448 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:30,448 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:30,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: A74D81EEBF854B3399989ED07D49E0DA, VertexParallelism: 1, TaskAttemptID:attempt_1452796290253_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E5010A52968349B2861DEA585CB100ED, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F17459C26ACC4A2783B8F069A238FF18, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:30,448 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:30,448 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796290253_0001_1_00_000000_0
2016-01-14 18:31:30,448 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:30,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:30,449 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796290253_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:30,449 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:30,451 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:30,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E5010A52968349B2861DEA585CB100ED, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:30,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:30,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E5010A52968349B2861DEA585CB100ED
2016-01-14 18:31:30,453 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E5010A52968349B2861DEA585CB100ED
2016-01-14 18:31:30,452 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F17459C26ACC4A2783B8F069A238FF18, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:30,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:30,454 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F17459C26ACC4A2783B8F069A238FF18
2016-01-14 18:31:30,455 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:30,458 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E5010A52968349B2861DEA585CB100ED
2016-01-14 18:31:30,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E5010A52968349B2861DEA585CB100ED
2016-01-14 18:31:30,456 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:30,458 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F17459C26ACC4A2783B8F069A238FF18
2016-01-14 18:31:30,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F17459C26ACC4A2783B8F069A238FF18
2016-01-14 18:31:30,457 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A74D81EEBF854B3399989ED07D49E0DA, ordinal: 0
2016-01-14 18:31:30,459 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A74D81EEBF854B3399989ED07D49E0DA, mem on start (mb), free: 283, total: 369, max: 672
2016-01-14 18:31:30,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:30,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:30,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:30,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:30,460 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:30,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:30,460 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:30,461 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:30,461 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:30,461 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:30,462 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E5010A52968349B2861DEA585CB100ED, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:30,462 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F17459C26ACC4A2783B8F069A238FF18, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:30,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:30,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E5010A52968349B2861DEA585CB100ED being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:30,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E5010A52968349B2861DEA585CB100ED
2016-01-14 18:31:30,463 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:30,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E5010A52968349B2861DEA585CB100ED
2016-01-14 18:31:30,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:30,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:30,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:30,464 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796290253_0001_1_00_000000_0
2016-01-14 18:31:30,464 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:30,464 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@67245a95
2016-01-14 18:31:30,465 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@67245a95
2016-01-14 18:31:30,465 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:30,465 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:30,468 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:30,468 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E5010A52968349B2861DEA585CB100ED
2016-01-14 18:31:30,469 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 2096C7A9F1384ECA812C374BE53E19DE
2016-01-14 18:31:30,469 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 2096C7A9F1384ECA812C374BE53E19DE
2016-01-14 18:31:30,470 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@440bda03; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@66161ddb; comparator=cascading.tuple.hadoop.util.TupleComparator@36024939; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:30,470 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:30,470 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:30,470 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:30,484 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:30,484 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:30,484 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:30,484 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:30,485 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:30,485 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:30,485 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:30,486 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A74D81EEBF854B3399989ED07D49E0DA, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:30,489 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796290253_0001_1_00_000000_0
2016-01-14 18:31:30,489 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A74D81EEBF854B3399989ED07D49E0DA, mem on close (mb), free: 181, total: 369, max: 672
2016-01-14 18:31:30,489 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:30,489 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:30,489 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:30,489 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:30,491 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:30 Completed running task attempt: attempt_1452796290253_0001_1_00_000000_0
2016-01-14 18:31:30,496 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796290253_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:30,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796290486, Process_Duration=3, Process_End_Time=1452796290489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:30,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796290253_0001_1_00, eventCount=1
2016-01-14 18:31:30,497 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:30,497 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:30,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A74D81EEBF854B3399989ED07D49E0DA, taskAttemptId=attempt_1452796290253_0001_1_00_000000_0, startTime=1452796290445, finishTime=1452796290497, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796290486, Process_Duration=3, Process_End_Time=1452796290489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:30,498 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290253_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:30,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:30,498 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796290253_0001_00_000001
2016-01-14 18:31:30,498 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:30,498 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796290253_0001_1_00_000000_0
2016-01-14 18:31:30,499 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:TASK_FINISHED]: vertexName=A74D81EEBF854B3399989ED07D49E0DA, taskId=task_1452796290253_0001_1_00_000000, startTime=1452796290445, finishTime=1452796290499, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452796290253_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796290486, Process_Duration=3, Process_End_Time=1452796290489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:30,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:30,499 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290253_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:30,499 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA] : 1
2016-01-14 18:31:30,500 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA]
2016-01-14 18:31:30,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:VERTEX_FINISHED]: vertexName=A74D81EEBF854B3399989ED07D49E0DA, vertexId=vertex_1452796290253_0001_1_00, initRequestedTime=1452796290418, initedTime=1452796290432, startRequestedTime=1452796290433, startedTime=1452796290433, finishTime=1452796290500, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796290486, Process_Duration=3, Process_End_Time=1452796290489, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796290445, firstTasksToStart=[ task_1452796290253_0001_1_00_000000 ], lastTaskFinishTime=1452796290497, lastTasksToFinish=[ task_1452796290253_0001_1_00_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796290253_0001_1_00_000000 ], longestDurationTasks=[ task_1452796290253_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:30,500 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:30,501 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:30,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796290253_0001_00_000001, stoppedTime=1452796290501, exitStatus=0
2016-01-14 18:31:30,501 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:30,501 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18] attempt: attempt_1452796290253_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:30,501 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F17459C26ACC4A2783B8F069A238FF18
2016-01-14 18:31:30,501 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18]
2016-01-14 18:31:30,501 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:30,501 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796290253_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:30,501 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:TASK_STARTED]: vertexName=F17459C26ACC4A2783B8F069A238FF18, taskId=task_1452796290253_0001_1_01_000000, scheduledTime=1452796290501, launchTime=1452796290501
2016-01-14 18:31:30,502 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290253_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:30,502 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:30,502 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18]
2016-01-14 18:31:30,502 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: F17459C26ACC4A2783B8F069A238FF18, VertexParallelism: 1, TaskAttemptID:attempt_1452796290253_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A74D81EEBF854B3399989ED07D49E0DA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C59DFDC26FDB456597B918E5F869E910, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:30,502 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290253_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:30,502 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:30,503 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796290253_0001_00_000002
2016-01-14 18:31:30,503 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:30,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:30,505 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796290253_0001_00_000002
2016-01-14 18:31:30,505 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:30,505 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:30,505 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:30,505 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:30,506 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:30,506 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796290253_0001_00_000002, launchTime=1452796290506
2016-01-14 18:31:30,507 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796290253_0001_00_000002
2016-01-14 18:31:30,507 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796290253_0001_1_01_000000_0] to container: [container_1452796290253_0001_00_000002]
2016-01-14 18:31:30,507 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:30,508 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796290253_0001_1_01_000000_0] started. Is using containerId: [container_1452796290253_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:30,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F17459C26ACC4A2783B8F069A238FF18, taskAttemptId=attempt_1452796290253_0001_1_01_000000_0, startTime=1452796290508, containerId=container_1452796290253_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796290253_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:30,508 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290253_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:30,508 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290253_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:30,508 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:30,508 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796290253_0001_00_000002 given task: attempt_1452796290253_0001_1_01_000000_0
2016-01-14 18:31:30,508 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796290253_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:30,509 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796290253_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:30,508 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796290253_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796290253_0001_1_01_000000_0
2016-01-14 18:31:30,509 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796290253_0001_00_000001, Exiting
2016-01-14 18:31:30,509 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796290253_0001_00_000001
2016-01-14 18:31:30,509 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796290253_0001_00_000001
2016-01-14 18:31:30 Starting to run new task attempt: attempt_1452796290253_0001_1_01_000000_0
2016-01-14 18:31:30,509 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796290253_0001_1_01_000000_0
2016-01-14 18:31:30,509 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:30,509 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:30,509 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:30,509 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:30,509 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:30,510 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:30,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8] every/(1/1) ...lyplatform/0_'field'_SWAP, VertexName: F17459C26ACC4A2783B8F069A238FF18, VertexParallelism: 1, TaskAttemptID:attempt_1452796290253_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A74D81EEBF854B3399989ED07D49E0DA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C59DFDC26FDB456597B918E5F869E910, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:30,510 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:30,510 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796290253_0001_1_01_000000_0
2016-01-14 18:31:30,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:30,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:30,510 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796290253_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:30,511 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796290253_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:30,511 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796290253_0001_00_000001 completed successfully
2016-01-14 18:31:30,511 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A74D81EEBF854B3399989ED07D49E0DA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:30,511 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C59DFDC26FDB456597B918E5F869E910, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:30,511 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:30,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:30,511 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:30,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C59DFDC26FDB456597B918E5F869E910
2016-01-14 18:31:30,512 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:30,512 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C59DFDC26FDB456597B918E5F869E910
2016-01-14 18:31:30,515 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:30,512 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:30,516 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A74D81EEBF854B3399989ED07D49E0DA
2016-01-14 18:31:30,519 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:30,519 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:30,519 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A74D81EEBF854B3399989ED07D49E0DA
2016-01-14 18:31:30,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A74D81EEBF854B3399989ED07D49E0DA
2016-01-14 18:31:30,521 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F17459C26ACC4A2783B8F069A238FF18, ordinal: 1
2016-01-14 18:31:30,522 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F17459C26ACC4A2783B8F069A238FF18, mem on start (mb), free: 175, total: 369, max: 672
2016-01-14 18:31:30,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:30,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:30,522 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:30,522 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:30,523 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C59DFDC26FDB456597B918E5F869E910
2016-01-14 18:31:30,523 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:30,523 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:30,523 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:30,523 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:30,523 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:30,523 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:30,524 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:30,524 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C59DFDC26FDB456597B918E5F869E910, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:30,524 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A74D81EEBF854B3399989ED07D49E0DA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:30,524 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:30,524 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A74D81EEBF854B3399989ED07D49E0DA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:30,524 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:30,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:30,524 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A74D81EEBF854B3399989ED07D49E0DA
2016-01-14 18:31:30,525 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:30,525 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:30,526 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A74D81EEBF854B3399989ED07D49E0DA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:30,526 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:30,526 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:30,526 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:30,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A74D81EEBF854B3399989ED07D49E0DA
2016-01-14 18:31:30,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:30,527 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796290253_0001_1_01_000000_0
2016-01-14 18:31:30,533 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796290253_0001_1_00_000000_0_10141, runDuration: 40880, ]
2016-01-14 18:31:30,535 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796290253_0001_1_00_000000_0_10141] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796290535, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:30,535 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A74D81EEBF854B3399989ED07D49E0DA
2016-01-14 18:31:30,535 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:30,536 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:30,536 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 2096C7A9F1384ECA812C374BE53E19DE
2016-01-14 18:31:30,536 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP"], id: C59DFDC26FDB456597B918E5F869E910
2016-01-14 18:31:30,536 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_'field'_SWAP"]
2016-01-14 18:31:30,537 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 2096C7A9F1384ECA812C374BE53E19DE, for 1 inputs
2016-01-14 18:31:30,536 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A74D81EEBF854B3399989ED07D49E0DA] #24 in 2ms
2016-01-14 18:31:30,537 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A74D81EEBF854B3399989ED07D49E0DA] #24, status:false, isInterrupted:false
2016-01-14 18:31:30,537 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:30,538 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:30,538 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:30,538 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:30,538 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:30,538 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A74D81EEBF854B3399989ED07D49E0DA
2016-01-14 18:31:30,538 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:30,539 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F17459C26ACC4A2783B8F069A238FF18, all 1 inputs ready in: 00:00:00.002
2016-01-14 18:31:30,539 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@245944e6; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@60ba6aae
2016-01-14 18:31:30,543 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:30,543 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:30,543 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796290253_0001_1_01_000000_0
2016-01-14 18:31:30,543 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796290253_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:30,544 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796290253_0001_1_01_000000_0
2016-01-14 18:31:30,544 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F17459C26ACC4A2783B8F069A238FF18, mem on close (mb), free: 172, total: 369, max: 672
2016-01-14 18:31:30,544 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A74D81EEBF854B3399989ED07D49E0DA
2016-01-14 18:31:30,545 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:30,545 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:30,545 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:30,545 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:30,545 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:30 Completed running task attempt: attempt_1452796290253_0001_1_01_000000_0
2016-01-14 18:31:30,546 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796290253_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:30,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:30,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F17459C26ACC4A2783B8F069A238FF18, taskAttemptId=attempt_1452796290253_0001_1_01_000000_0, startTime=1452796290508, finishTime=1452796290547, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796290539, Process_Duration=5, Process_End_Time=1452796290544, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:30,547 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290253_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:30,547 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:30,547 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796290253_0001_1_01_000000_0
2016-01-14 18:31:30,548 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:30,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:TASK_FINISHED]: vertexName=F17459C26ACC4A2783B8F069A238FF18, taskId=task_1452796290253_0001_1_01_000000, startTime=1452796290508, finishTime=1452796290548, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1452796290253_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796290539, Process_Duration=5, Process_End_Time=1452796290544, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:30,548 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290253_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:30,548 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18] : 1
2016-01-14 18:31:30,548 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18]
2016-01-14 18:31:30,549 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:30,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:VERTEX_FINISHED]: vertexName=F17459C26ACC4A2783B8F069A238FF18, vertexId=vertex_1452796290253_0001_1_01, initRequestedTime=1452796290419, initedTime=1452796290425, startRequestedTime=1452796290437, startedTime=1452796290437, finishTime=1452796290549, timeTaken=112, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796290539, Process_Duration=5, Process_End_Time=1452796290544, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796290508, firstTasksToStart=[ task_1452796290253_0001_1_01_000000 ], lastTaskFinishTime=1452796290547, lastTasksToFinish=[ task_1452796290253_0001_1_01_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796290253_0001_1_01_000000 ], longestDurationTasks=[ task_1452796290253_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:30,550 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:30,550 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:30,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796290253_0001_00_000002, stoppedTime=1452796290550, exitStatus=0
2016-01-14 18:31:30,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:30,550 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:30,550 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:30,551 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796290253_0001_1
2016-01-14 18:31:30,551 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796290253_0001_1
2016-01-14 18:31:30,551 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796290253_0001_1_00 [A74D81EEBF854B3399989ED07D49E0DA]
2016-01-14 18:31:30,551 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: C59DFDC26FDB456597B918E5F869E910 for vertex: vertex_1452796290253_0001_1_01 [F17459C26ACC4A2783B8F069A238FF18]
2016-01-14 18:31:30,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387448832, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796290539, Process_Duration=5, Process_End_Time=1452796290544, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:30,552 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:30,553 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:30,553 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796290253_0001_00_000002
2016-01-14 18:31:30,553 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796290253_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:30,554 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796290253_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:30,554 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796290253_0001_00_000002, Exiting
2016-01-14 18:31:30,554 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796290253_0001_00_000002
2016-01-14 18:31:30,554 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796290253_0001_00_000002
2016-01-14 18:31:30,554 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796290253_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:30,554 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796290253_0001_00_000002 completed successfully
2016-01-14 18:31:30,561 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796290253_0001_1, queueSize=0
2016-01-14 18:31:30,562 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:30,562 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290253_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796290253_0001_1, startTime=1452796290416, finishTime=1452796290559, timeTaken=143, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=516, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=774897664, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=8, LAST_EVENT_RECEIVED=8, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592581025, Process_Duration=8, Process_End_Time=2905592581033, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:30,562 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796290253_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:30,562 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796290253_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:30,562 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290253_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:30,562 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796290253_0001_00_000002
2016-01-14 18:31:30,562 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:30,562 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:30 Completed Dag: dag_1452796290253_0001_1
2016-01-14 18:31:30,562 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796290253_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:30,563 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796290253_0001_1_post
2016-01-14 18:31:30,563 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:30,563 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:30,569 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_'field'_SWAP, applicationId=application_1452796290253_0001
2016-01-14 18:31:30,571 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:30,571 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:30,571 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:30,571 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:30,571 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:30,571 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:30,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:30,572 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:30,572 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:30,572 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:30,572 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:30,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:30,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:30,577 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/609E680CE4CF4E8D941B83772B11EEB7/A477F21C6B1C43F3A2AF0A11B41441C8/.tez/application_1452796290253_0001
2016-01-14 18:31:30,578 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.344
2016-01-14 18:31:30,644 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_'field'_SWAP
2016-01-14 18:31:30,666 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:30,667 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:30,702 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 18:31:30,709 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 18:31:30,709 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:30,709 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:30,710 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:30,724 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:30,729 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1
2016-01-14 18:31:30,798 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:30,798 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:30,798 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1"]
2016-01-14 18:31:30,798 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:30,798 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:30,799 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:30,799 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...mblyplatform/0_UNKNOWN_-1
2016-01-14 18:31:30,799 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:30,812 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:30,816 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:30,817 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:30,817 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:30,825 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0/.tez/application_1452796290816_0001 doesn't exist and is created
2016-01-14 18:31:30,838 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:30,838 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:30,839 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0/.tez/application_1452796290816_0001
2016-01-14 18:31:30,841 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796290816_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:30,852 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:30,853 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:30,854 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:30,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:30,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0/.tez/application_1452796290816_0001/localmode-log-dir
2016-01-14 18:31:30,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0/.tez/application_1452796290816_0001/localmode-log-dir/history.txt.appattempt_1452796290816_0001_000000, maxErrors=10
2016-01-14 18:31:30,857 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:30,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796290816_0001_000000, appSubmitTime=1452796290841, launchTime=1452796290841
2016-01-14 18:31:30,860 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:30,861 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:30,865 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:30,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:30,867 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:54822
2016-01-14 18:31:30,869 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:30,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796290816_0001_000000, startTime=1452796290870
2016-01-14 18:31:30,870 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:30,870 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:30,939 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:30,939 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:30,939 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1452796290816_0001, dagName=[990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-01-14 18:31:30,954 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-01-14 18:31:30,956 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796290816_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0/.tez/application_1452796290816_0001/localmode-log-dir/dag_1452796290816_0001_1.dot
2016-01-14 18:31:30,956 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0/.tez/application_1452796290816_0001/localmode-log-dir/dag_1452796290816_0001_1-tez-dag.pb.txt
2016-01-14 18:31:30,962 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796290816_0001_1
2016-01-14 18:31:30,963 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:30,963 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-01-14 18:31:30 Running Dag: dag_1452796290816_0001_1
2016-01-14 18:31:30,973 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796290816_0001_1, submitTime=1452796290954
2016-01-14 18:31:30,973 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:30,974 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E9B3161D5FFF4E0D8D077316479778CD
2016-01-14 18:31:30,974 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:30,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796290816_0001_1, initTime=1452796290974
2016-01-14 18:31:30,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:30,975 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796290816_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:30,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:DAG_STARTED]: dagID=dag_1452796290816_0001_1, startTime=1452796290975
2016-01-14 18:31:30,975 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:30,976 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796290816_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:30,976 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1452796290816_0001, dagName=[990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0] every/(1/1) ...mblyplatform/0_UNKNOWN_-1
2016-01-14 18:31:30,976 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:30,976 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C5555BF5876141F398ADECF7E631484E : {F7EA4CA4BFE944FBB0DABA9D9067DDCD={InputName=F7EA4CA4BFE944FBB0DABA9D9067DDCD}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:30,977 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F7EA4CA4BFE944FBB0DABA9D9067DDCD, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:30,977 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E]
2016-01-14 18:31:30,977 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E]
2016-01-14 18:31:30,977 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E]
2016-01-14 18:31:30,978 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E]. Starting root input initializers: 1
2016-01-14 18:31:30,978 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:30,978 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD]
2016-01-14 18:31:30,978 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F7EA4CA4BFE944FBB0DABA9D9067DDCD on vertex vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E]
2016-01-14 18:31:30,980 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:30,981 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F7EA4CA4BFE944FBB0DABA9D9067DDCD asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:30,981 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:30,981 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:30,981 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:30,982 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD]
2016-01-14 18:31:30,982 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD]
2016-01-14 18:31:30,982 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD]
2016-01-14 18:31:30,982 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=96B9B50C11A94201818BE117250F083C, vertexId=vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:30,982 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=96B9B50C11A94201818BE117250F083C, vertexId=vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD]
2016-01-14 18:31:30,983 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:30,983 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F7EA4CA4BFE944FBB0DABA9D9067DDCD on vertex vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E]
2016-01-14 18:31:30,985 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=96B9B50C11A94201818BE117250F083C, vertexName=E9B3161D5FFF4E0D8D077316479778CD, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:30,985 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=96B9B50C11A94201818BE117250F083C, vertexId=vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD]
2016-01-14 18:31:30,986 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:30,986 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E9B3161D5FFF4E0D8D077316479778CD, vertexId=vertex_1452796290816_0001_1_01, initRequestedTime=1452796290978, initedTime=1452796290986, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:30,986 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:30,986 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:30,986 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F7EA4CA4BFE944FBB0DABA9D9067DDCD=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:30,987 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E] parallelism set to 1
2016-01-14 18:31:30,987 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:30,987 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796290816_0001_1_00, eventCount=1
2016-01-14 18:31:30,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C5555BF5876141F398ADECF7E631484E, vertexId=vertex_1452796290816_0001_1_00, initRequestedTime=1452796290976, initedTime=1452796290988, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:30,988 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E] with distanceFromRoot: 0
2016-01-14 18:31:30,988 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:30,988 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:30,989 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C5555BF5876141F398ADECF7E631484E
2016-01-14 18:31:30,989 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E]
2016-01-14 18:31:30,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796290816_0001_1_00, startRequestedTime=1452796290988, startedTime=1452796290988
2016-01-14 18:31:30,989 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:30,989 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:30,990 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:30,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:TASK_STARTED]: vertexName=C5555BF5876141F398ADECF7E631484E, taskId=task_1452796290816_0001_1_00_000000, scheduledTime=1452796290990, launchTime=1452796290990
2016-01-14 18:31:30,991 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290816_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:30,991 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796290816_0001_1_00 for vertex: vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD] numStartedSources: 1 numSources: 1
2016-01-14 18:31:30,991 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD] with distanceFromRoot: 1
2016-01-14 18:31:30,991 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E]
2016-01-14 18:31:30,992 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C5555BF5876141F398ADECF7E631484E in vertex: E9B3161D5FFF4E0D8D077316479778CD
2016-01-14 18:31:30,992 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E9B3161D5FFF4E0D8D077316479778CD with 1 source tasks and 1 pending tasks
2016-01-14 18:31:30,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:30,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796290816_0001_1_01, startRequestedTime=1452796290991, startedTime=1452796290991
2016-01-14 18:31:30,993 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:30,993 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796290816_0001, with dag id: dag_1452796290816_0001_1
2016-01-14 18:31:30,993 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: C5555BF5876141F398ADECF7E631484E, VertexParallelism: 1, TaskAttemptID:attempt_1452796290816_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7EA4CA4BFE944FBB0DABA9D9067DDCD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9B3161D5FFF4E0D8D077316479778CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:30,993 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290816_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:30,993 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:30,994 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:30,994 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:30,994 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:30,995 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796290816_0001_00_000001
2016-01-14 18:31:30,995 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:30,996 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796290816_0001_00_000001
2016-01-14 18:31:30,996 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:30,997 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:30,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796290816_0001_00_000001, launchTime=1452796290997
2016-01-14 18:31:30,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:30,998 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796290816_0001_00_000001
2016-01-14 18:31:30,998 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796290816_0001_1_00_000000_0] to container: [container_1452796290816_0001_00_000001]
2016-01-14 18:31:30,998 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:30,998 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796290816_0001_00_000001 given task: attempt_1452796290816_0001_1_00_000000_0
2016-01-14 18:31:30,999 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796290816_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796290816_0001_1_00_000000_0
2016-01-14 18:31:30,999 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796290816_0001_1_00_000000_0] started. Is using containerId: [container_1452796290816_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:30,999 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C5555BF5876141F398ADECF7E631484E, taskAttemptId=attempt_1452796290816_0001_1_00_000000_0, startTime=1452796290999, containerId=container_1452796290816_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796290816_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:30,999 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290816_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:30,999 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290816_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:30 Starting to run new task attempt: attempt_1452796290816_0001_1_00_000000_0
2016-01-14 18:31:30,999 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:31,000 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796290816_0001_1_00_000000_0
2016-01-14 18:31:31,000 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:31,000 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:31,000 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:31,001 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:31,001 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:31,001 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:31,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: C5555BF5876141F398ADECF7E631484E, VertexParallelism: 1, TaskAttemptID:attempt_1452796290816_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7EA4CA4BFE944FBB0DABA9D9067DDCD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E9B3161D5FFF4E0D8D077316479778CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:31,001 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:31,002 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796290816_0001_1_00_000000_0
2016-01-14 18:31:31,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:31,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,002 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796290816_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:31,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E9B3161D5FFF4E0D8D077316479778CD, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:31,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F7EA4CA4BFE944FBB0DABA9D9067DDCD, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:31,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:31,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F7EA4CA4BFE944FBB0DABA9D9067DDCD
2016-01-14 18:31:31,004 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F7EA4CA4BFE944FBB0DABA9D9067DDCD
2016-01-14 18:31:31,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:31,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E9B3161D5FFF4E0D8D077316479778CD
2016-01-14 18:31:31,006 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:31,006 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:31,006 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F7EA4CA4BFE944FBB0DABA9D9067DDCD
2016-01-14 18:31:31,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F7EA4CA4BFE944FBB0DABA9D9067DDCD
2016-01-14 18:31:31,007 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:31,007 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E9B3161D5FFF4E0D8D077316479778CD
2016-01-14 18:31:31,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E9B3161D5FFF4E0D8D077316479778CD
2016-01-14 18:31:31,010 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C5555BF5876141F398ADECF7E631484E, ordinal: 0
2016-01-14 18:31:31,010 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C5555BF5876141F398ADECF7E631484E, mem on start (mb), free: 279, total: 370, max: 672
2016-01-14 18:31:31,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:31,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:31,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:31,011 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:31,011 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:31,011 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:31,012 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:31,012 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:31,012 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:31,012 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F7EA4CA4BFE944FBB0DABA9D9067DDCD, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:31,013 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E9B3161D5FFF4E0D8D077316479778CD, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:31,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:31,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F7EA4CA4BFE944FBB0DABA9D9067DDCD being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:31,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F7EA4CA4BFE944FBB0DABA9D9067DDCD
2016-01-14 18:31:31,013 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:31,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F7EA4CA4BFE944FBB0DABA9D9067DDCD
2016-01-14 18:31:31,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:31,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:31,014 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:31,014 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796290816_0001_1_00_000000_0
2016-01-14 18:31:31,014 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:31,014 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3881ab73
2016-01-14 18:31:31,015 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3881ab73
2016-01-14 18:31:31,015 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:31,016 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:31,017 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:31,017 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F7EA4CA4BFE944FBB0DABA9D9067DDCD
2016-01-14 18:31:31,018 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 08416CB83EBE45549E9B211DA2584B18
2016-01-14 18:31:31,018 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 08416CB83EBE45549E9B211DA2584B18
2016-01-14 18:31:31,018 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1ef57653; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@52b5306; comparator=cascading.tuple.hadoop.util.TupleComparator@5f57b190; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:31,019 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:31,019 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:31,019 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:31,032 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:31,033 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:31,033 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:31,033 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:31,033 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:31,034 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:31,034 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:31,034 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C5555BF5876141F398ADECF7E631484E, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:31,037 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796290816_0001_1_00_000000_0
2016-01-14 18:31:31,037 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C5555BF5876141F398ADECF7E631484E, mem on close (mb), free: 177, total: 370, max: 672
2016-01-14 18:31:31,037 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:31,037 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:31,038 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:31,038 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:31,040 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:31 Completed running task attempt: attempt_1452796290816_0001_1_00_000000_0
2016-01-14 18:31:31,043 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796290816_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:31,043 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387973120, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796291034, Process_Duration=3, Process_End_Time=1452796291037, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:31,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796290816_0001_1_00, eventCount=1
2016-01-14 18:31:31,044 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C5555BF5876141F398ADECF7E631484E, taskAttemptId=attempt_1452796290816_0001_1_00_000000_0, startTime=1452796290999, finishTime=1452796291043, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387973120, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291034, Process_Duration=3, Process_End_Time=1452796291037, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:31,044 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290816_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:31,044 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:31,044 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:31,044 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796290816_0001_1_00_000000_0
2016-01-14 18:31:31,044 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:31,044 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:31,045 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796290816_0001_00_000001
2016-01-14 18:31:31,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:31,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:TASK_FINISHED]: vertexName=C5555BF5876141F398ADECF7E631484E, taskId=task_1452796290816_0001_1_00_000000, startTime=1452796290999, finishTime=1452796291044, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452796290816_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387973120, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291034, Process_Duration=3, Process_End_Time=1452796291037, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:31,045 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290816_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:31,045 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E] : 1
2016-01-14 18:31:31,045 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E]
2016-01-14 18:31:31,046 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:VERTEX_FINISHED]: vertexName=C5555BF5876141F398ADECF7E631484E, vertexId=vertex_1452796290816_0001_1_00, initRequestedTime=1452796290976, initedTime=1452796290988, startRequestedTime=1452796290988, startedTime=1452796290988, finishTime=1452796291045, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387973120, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291034, Process_Duration=3, Process_End_Time=1452796291037, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796290999, firstTasksToStart=[ task_1452796290816_0001_1_00_000000 ], lastTaskFinishTime=1452796291043, lastTasksToFinish=[ task_1452796290816_0001_1_00_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796290816_0001_1_00_000000 ], longestDurationTasks=[ task_1452796290816_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:31,046 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:31,046 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:31,047 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:31,047 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796290816_0001_00_000001, stoppedTime=1452796291047, exitStatus=0
2016-01-14 18:31:31,047 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:31,047 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD] attempt: attempt_1452796290816_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:31,047 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E9B3161D5FFF4E0D8D077316479778CD
2016-01-14 18:31:31,047 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:31,047 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD]
2016-01-14 18:31:31,048 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:31,048 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796290816_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:31,048 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:31,048 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:TASK_STARTED]: vertexName=E9B3161D5FFF4E0D8D077316479778CD, taskId=task_1452796290816_0001_1_01_000000, scheduledTime=1452796291048, launchTime=1452796291048
2016-01-14 18:31:31,048 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290816_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:31,048 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:31,048 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD]
2016-01-14 18:31:31,049 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: E9B3161D5FFF4E0D8D077316479778CD, VertexParallelism: 1, TaskAttemptID:attempt_1452796290816_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C5555BF5876141F398ADECF7E631484E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=96B9B50C11A94201818BE117250F083C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:31,049 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290816_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:31,049 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:31,050 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796290816_0001_00_000002
2016-01-14 18:31:31,050 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:31,052 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796290816_0001_00_000002
2016-01-14 18:31:31,052 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:31,052 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:31,052 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:31,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796290816_0001_00_000002, launchTime=1452796291052
2016-01-14 18:31:31,053 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796290816_0001_00_000002
2016-01-14 18:31:31,053 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796290816_0001_1_01_000000_0] to container: [container_1452796290816_0001_00_000002]
2016-01-14 18:31:31,053 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:31,054 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796290816_0001_1_01_000000_0] started. Is using containerId: [container_1452796290816_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:31,054 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E9B3161D5FFF4E0D8D077316479778CD, taskAttemptId=attempt_1452796290816_0001_1_01_000000_0, startTime=1452796291054, containerId=container_1452796290816_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796290816_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:31,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:31,054 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290816_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:31,054 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290816_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:31,054 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796290816_0001_00_000002 given task: attempt_1452796290816_0001_1_01_000000_0
2016-01-14 18:31:31,054 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796290816_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796290816_0001_1_01_000000_0
2016-01-14 18:31:31 Starting to run new task attempt: attempt_1452796290816_0001_1_01_000000_0
2016-01-14 18:31:31,055 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796290816_0001_1_01_000000_0
2016-01-14 18:31:31,055 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:31,055 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:31,055 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:31,055 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:31,055 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:31,056 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:31,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0] every/(1/1) ...mblyplatform/0_UNKNOWN_-1, VertexName: E9B3161D5FFF4E0D8D077316479778CD, VertexParallelism: 1, TaskAttemptID:attempt_1452796290816_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C5555BF5876141F398ADECF7E631484E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=96B9B50C11A94201818BE117250F083C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:31,056 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:31,055 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796290816_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:31,056 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796290816_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:31,056 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796290816_0001_00_000001, Exiting
2016-01-14 18:31:31,057 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796290816_0001_00_000001
2016-01-14 18:31:31,057 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796290816_0001_00_000001
2016-01-14 18:31:31,057 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796290816_0001_1_01_000000_0
2016-01-14 18:31:31,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:31,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,057 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796290816_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:31,058 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796290816_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:31,058 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796290816_0001_00_000001 completed successfully
2016-01-14 18:31:31,058 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:31,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C5555BF5876141F398ADECF7E631484E, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:31,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=96B9B50C11A94201818BE117250F083C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:31,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:31,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C5555BF5876141F398ADECF7E631484E
2016-01-14 18:31:31,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:31,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 96B9B50C11A94201818BE117250F083C
2016-01-14 18:31:31,061 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:31,061 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 96B9B50C11A94201818BE117250F083C
2016-01-14 18:31:31,062 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:31,063 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:31,064 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:31,065 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C5555BF5876141F398ADECF7E631484E
2016-01-14 18:31:31,065 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C5555BF5876141F398ADECF7E631484E
2016-01-14 18:31:31,069 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:31,069 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E9B3161D5FFF4E0D8D077316479778CD, ordinal: 1
2016-01-14 18:31:31,069 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E9B3161D5FFF4E0D8D077316479778CD, mem on start (mb), free: 171, total: 370, max: 672
2016-01-14 18:31:31,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 96B9B50C11A94201818BE117250F083C
2016-01-14 18:31:31,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:31,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:31,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:31,070 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:31,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:31,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:31,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:31,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:31,070 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:31,070 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 96B9B50C11A94201818BE117250F083C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:31,070 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C5555BF5876141F398ADECF7E631484E, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:31,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:31,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C5555BF5876141F398ADECF7E631484E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:31,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C5555BF5876141F398ADECF7E631484E
2016-01-14 18:31:31,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:31,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:31,071 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:31,071 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:31,072 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C5555BF5876141F398ADECF7E631484E with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:31,072 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:31,072 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:31,072 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:31,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C5555BF5876141F398ADECF7E631484E
2016-01-14 18:31:31,073 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:31,073 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796290816_0001_1_01_000000_0
2016-01-14 18:31:31,074 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796290816_0001_1_00_000000_0_10146, runDuration: 36815, ]
2016-01-14 18:31:31,076 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:31,077 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 08416CB83EBE45549E9B211DA2584B18
2016-01-14 18:31:31,077 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1"], id: 96B9B50C11A94201818BE117250F083C
2016-01-14 18:31:31,077 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_-1"]
2016-01-14 18:31:31,078 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 08416CB83EBE45549E9B211DA2584B18, for 1 inputs
2016-01-14 18:31:31,077 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796290816_0001_1_00_000000_0_10146] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796291075, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:31,079 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C5555BF5876141F398ADECF7E631484E
2016-01-14 18:31:31,079 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:31,079 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C5555BF5876141F398ADECF7E631484E] #25 in 5ms
2016-01-14 18:31:31,079 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C5555BF5876141F398ADECF7E631484E] #25, status:false, isInterrupted:false
2016-01-14 18:31:31,080 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:31,080 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:31,080 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:31,080 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:31,080 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:31,081 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C5555BF5876141F398ADECF7E631484E
2016-01-14 18:31:31,081 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:31,081 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E9B3161D5FFF4E0D8D077316479778CD, all 1 inputs ready in: 00:00:00.003
2016-01-14 18:31:31,081 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@43c60298; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7cd46bea
2016-01-14 18:31:31,084 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:31,085 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:31,085 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796290816_0001_1_01_000000_0
2016-01-14 18:31:31,085 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796290816_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:31,086 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796290816_0001_1_01_000000_0
2016-01-14 18:31:31,086 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E9B3161D5FFF4E0D8D077316479778CD, mem on close (mb), free: 167, total: 370, max: 672
2016-01-14 18:31:31,086 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C5555BF5876141F398ADECF7E631484E
2016-01-14 18:31:31,086 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:31,087 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:31,087 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:31,087 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:31,087 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:31 Completed running task attempt: attempt_1452796290816_0001_1_01_000000_0
2016-01-14 18:31:31,088 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796290816_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:31,088 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:31,088 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E9B3161D5FFF4E0D8D077316479778CD, taskAttemptId=attempt_1452796290816_0001_1_01_000000_0, startTime=1452796291054, finishTime=1452796291088, timeTaken=34, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387973120, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291081, Process_Duration=5, Process_End_Time=1452796291086, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:31,089 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796290816_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:31,089 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796290816_0001_1_01_000000_0
2016-01-14 18:31:31,089 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:31,089 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:TASK_FINISHED]: vertexName=E9B3161D5FFF4E0D8D077316479778CD, taskId=task_1452796290816_0001_1_01_000000, startTime=1452796291054, finishTime=1452796291089, timeTaken=35, status=SUCCEEDED, successfulAttemptID=attempt_1452796290816_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387973120, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291081, Process_Duration=5, Process_End_Time=1452796291086, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:31,090 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796290816_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:31,090 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD] : 1
2016-01-14 18:31:31,090 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:31,090 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD]
2016-01-14 18:31:31,091 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:VERTEX_FINISHED]: vertexName=E9B3161D5FFF4E0D8D077316479778CD, vertexId=vertex_1452796290816_0001_1_01, initRequestedTime=1452796290978, initedTime=1452796290986, startRequestedTime=1452796290991, startedTime=1452796290991, finishTime=1452796291090, timeTaken=99, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387973120, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291081, Process_Duration=5, Process_End_Time=1452796291086, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796291054, firstTasksToStart=[ task_1452796290816_0001_1_01_000000 ], lastTaskFinishTime=1452796291088, lastTasksToFinish=[ task_1452796290816_0001_1_01_000000 ], minTaskDuration=34, maxTaskDuration=34, avgTaskDuration=34.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796290816_0001_1_01_000000 ], longestDurationTasks=[ task_1452796290816_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:31,091 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:31,091 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:31,093 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:31,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=387973120, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796291081, Process_Duration=5, Process_End_Time=1452796291086, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:31,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796290816_0001_00_000002, stoppedTime=1452796291093, exitStatus=0
2016-01-14 18:31:31,094 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:31,094 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:31,094 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:31,094 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796290816_0001_00_000002
2016-01-14 18:31:31,094 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:31,095 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796290816_0001_1
2016-01-14 18:31:31,094 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796290816_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:31,095 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796290816_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:31,095 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796290816_0001_00_000002, Exiting
2016-01-14 18:31:31,095 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796290816_0001_00_000002
2016-01-14 18:31:31,095 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796290816_0001_00_000002
2016-01-14 18:31:31,094 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:31,095 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796290816_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:31,096 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796290816_0001_00_000002 completed successfully
2016-01-14 18:31:31,095 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796290816_0001_1
2016-01-14 18:31:31,096 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 96B9B50C11A94201818BE117250F083C for vertex: vertex_1452796290816_0001_1_01 [E9B3161D5FFF4E0D8D077316479778CD]
2016-01-14 18:31:31,103 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796290816_0001_1_00 [C5555BF5876141F398ADECF7E631484E]
2016-01-14 18:31:31,104 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796290816_0001_1, queueSize=0
2016-01-14 18:31:31,105 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:31,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796290816_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796290816_0001_1, startTime=1452796290975, finishTime=1452796291103, timeTaken=128, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=775946240, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592582115, Process_Duration=8, Process_End_Time=2905592582123, Read_Duration=0, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:31,105 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796290816_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:31,105 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796290816_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:31,105 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796290816_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:31,105 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796290816_0001_00_000002
2016-01-14 18:31:31 Completed Dag: dag_1452796290816_0001_1
2016-01-14 18:31:31,105 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796290816_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:31,106 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796290816_0001_1_post
2016-01-14 18:31:31,106 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:31,106 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:31,113 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_UNKNOWN_-1, applicationId=application_1452796290816_0001
2016-01-14 18:31:31,115 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:31,115 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:31,115 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:31,115 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:31,116 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:31,116 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:31,116 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:31,116 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:31,116 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:31,117 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:31,117 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:31,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:31,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:31,122 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/990F94AED5424734B1E32F25E892958D/D2F5532324DD4A72ABF17918156AD6B0/.tez/application_1452796290816_0001
2016-01-14 18:31:31,123 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.324
2016-01-14 18:31:31,196 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_-1
2016-01-14 18:31:31,217 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:31,217 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:31,243 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 18:31:31,246 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.005
2016-01-14 18:31:31,246 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:31,246 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:31,246 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_'field'
2016-01-14 18:31:31,267 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:31,268 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:31,302 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:31,302 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:31,311 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.019
2016-01-14 18:31:31,315 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.024
2016-01-14 18:31:31,315 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:31,316 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:31,316 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:31,333 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:31,339 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS
2016-01-14 18:31:31,386 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:31,386 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:31,386 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS"]
2016-01-14 18:31:31,386 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:31,386 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:31,386 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:31,387 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/0_UNKNOWN_RESULTS
2016-01-14 18:31:31,387 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:31,399 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:31,403 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:31,403 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:31,403 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:31,412 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D/.tez/application_1452796291403_0001 doesn't exist and is created
2016-01-14 18:31:31,428 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:31,428 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:31,429 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D/.tez/application_1452796291403_0001
2016-01-14 18:31:31,430 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796291403_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:31,440 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:31,441 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:31,442 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:31,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:31,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D/.tez/application_1452796291403_0001/localmode-log-dir
2016-01-14 18:31:31,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D/.tez/application_1452796291403_0001/localmode-log-dir/history.txt.appattempt_1452796291403_0001_000000, maxErrors=10
2016-01-14 18:31:31,443 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:31,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796291403_0001_000000, appSubmitTime=1452796291430, launchTime=1452796291430
2016-01-14 18:31:31,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:31,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:31,450 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:31,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:31,454 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59305
2016-01-14 18:31:31,455 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:31,457 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796291403_0001_000000, startTime=1452796291457
2016-01-14 18:31:31,457 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:31,457 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:31,528 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:31,528 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:31,528 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1452796291403_0001, dagName=[8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-01-14 18:31:31,545 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-01-14 18:31:31,546 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796291403_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D/.tez/application_1452796291403_0001/localmode-log-dir/dag_1452796291403_0001_1.dot
2016-01-14 18:31:31,546 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D/.tez/application_1452796291403_0001/localmode-log-dir/dag_1452796291403_0001_1-tez-dag.pb.txt
2016-01-14 18:31:31,552 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796291403_0001_1
2016-01-14 18:31:31,553 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:31,553 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-01-14 18:31:31 Running Dag: dag_1452796291403_0001_1
2016-01-14 18:31:31,562 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:31,562 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796291403_0001_1, submitTime=1452796291545
2016-01-14 18:31:31,563 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DD09C6ACAEFE4560BF999FE739E88C09
2016-01-14 18:31:31,563 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:31,563 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:31,563 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796291403_0001_1, initTime=1452796291562
2016-01-14 18:31:31,563 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796291403_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:31,564 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:DAG_STARTED]: dagID=dag_1452796291403_0001_1, startTime=1452796291563
2016-01-14 18:31:31,564 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:31,564 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:31,564 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796291403_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:31,564 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E98F2153CB714A82ADF2F7BCB6C01446 : {16172E1960714427819A13672CAA68B0={InputName=16172E1960714427819A13672CAA68B0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:31,564 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 16172E1960714427819A13672CAA68B0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:31,564 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446]
2016-01-14 18:31:31,564 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1452796291403_0001, dagName=[8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D] every/(1/1) ...latform/0_UNKNOWN_RESULTS
2016-01-14 18:31:31,564 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446]
2016-01-14 18:31:31,564 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446]
2016-01-14 18:31:31,564 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446]. Starting root input initializers: 1
2016-01-14 18:31:31,565 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:31,565 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09]
2016-01-14 18:31:31,567 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 16172E1960714427819A13672CAA68B0 on vertex vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446]
2016-01-14 18:31:31,568 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:31,568 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09]
2016-01-14 18:31:31,569 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09]
2016-01-14 18:31:31,569 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09]
2016-01-14 18:31:31,569 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=6EF8925BE6EA4FF5ABEBC00C07B447D4, vertexId=vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:31,569 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=6EF8925BE6EA4FF5ABEBC00C07B447D4, vertexId=vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09]
2016-01-14 18:31:31,571 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:31,571 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=6EF8925BE6EA4FF5ABEBC00C07B447D4, vertexName=DD09C6ACAEFE4560BF999FE739E88C09, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:31,571 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=6EF8925BE6EA4FF5ABEBC00C07B447D4, vertexId=vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09]
2016-01-14 18:31:31,572 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 16172E1960714427819A13672CAA68B0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:31,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:31,572 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:31,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD09C6ACAEFE4560BF999FE739E88C09, vertexId=vertex_1452796291403_0001_1_01, initRequestedTime=1452796291565, initedTime=1452796291572, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:31,573 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:31,573 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:31,574 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:31,575 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 16172E1960714427819A13672CAA68B0 on vertex vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446]
2016-01-14 18:31:31,575 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:31,575 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {16172E1960714427819A13672CAA68B0=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:31,575 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446] parallelism set to 1
2016-01-14 18:31:31,576 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:31,576 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796291403_0001_1_00, eventCount=1
2016-01-14 18:31:31,576 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:31,576 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E98F2153CB714A82ADF2F7BCB6C01446, vertexId=vertex_1452796291403_0001_1_00, initRequestedTime=1452796291564, initedTime=1452796291576, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:31,577 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446] with distanceFromRoot: 0
2016-01-14 18:31:31,577 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:31,577 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E98F2153CB714A82ADF2F7BCB6C01446
2016-01-14 18:31:31,577 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446]
2016-01-14 18:31:31,578 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796291403_0001_1_00, startRequestedTime=1452796291577, startedTime=1452796291577
2016-01-14 18:31:31,578 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:31,578 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:31,579 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:31,579 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:TASK_STARTED]: vertexName=E98F2153CB714A82ADF2F7BCB6C01446, taskId=task_1452796291403_0001_1_00_000000, scheduledTime=1452796291579, launchTime=1452796291579
2016-01-14 18:31:31,579 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291403_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:31,579 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796291403_0001_1_00 for vertex: vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09] numStartedSources: 1 numSources: 1
2016-01-14 18:31:31,580 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09] with distanceFromRoot: 1
2016-01-14 18:31:31,580 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446]
2016-01-14 18:31:31,580 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E98F2153CB714A82ADF2F7BCB6C01446 in vertex: DD09C6ACAEFE4560BF999FE739E88C09
2016-01-14 18:31:31,581 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DD09C6ACAEFE4560BF999FE739E88C09 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:31,581 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796291403_0001_1_01, startRequestedTime=1452796291580, startedTime=1452796291580
2016-01-14 18:31:31,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:31,581 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:31,582 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: E98F2153CB714A82ADF2F7BCB6C01446, VertexParallelism: 1, TaskAttemptID:attempt_1452796291403_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=16172E1960714427819A13672CAA68B0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD09C6ACAEFE4560BF999FE739E88C09, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:31,582 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291403_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:31,582 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:31,583 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796291403_0001, with dag id: dag_1452796291403_0001_1
2016-01-14 18:31:31,583 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:31,583 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:31,583 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:31,584 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796291403_0001_00_000001
2016-01-14 18:31:31,584 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:31,586 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796291403_0001_00_000001
2016-01-14 18:31:31,586 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:31,586 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:31,586 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:31,588 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796291403_0001_00_000001, launchTime=1452796291586
2016-01-14 18:31:31,588 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796291403_0001_00_000001
2016-01-14 18:31:31,588 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796291403_0001_1_00_000000_0] to container: [container_1452796291403_0001_00_000001]
2016-01-14 18:31:31,588 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:31,589 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796291403_0001_00_000001 given task: attempt_1452796291403_0001_1_00_000000_0
2016-01-14 18:31:31,589 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796291403_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796291403_0001_1_00_000000_0
2016-01-14 18:31:31,589 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796291403_0001_1_00_000000_0] started. Is using containerId: [container_1452796291403_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:31,589 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:31,589 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E98F2153CB714A82ADF2F7BCB6C01446, taskAttemptId=attempt_1452796291403_0001_1_00_000000_0, startTime=1452796291589, containerId=container_1452796291403_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796291403_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:31,590 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291403_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:31,590 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291403_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:31 Starting to run new task attempt: attempt_1452796291403_0001_1_00_000000_0
2016-01-14 18:31:31,590 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796291403_0001_1_00_000000_0
2016-01-14 18:31:31,590 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:31,590 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:31,590 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:31,593 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:31,593 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:31,594 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:31,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: E98F2153CB714A82ADF2F7BCB6C01446, VertexParallelism: 1, TaskAttemptID:attempt_1452796291403_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=16172E1960714427819A13672CAA68B0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD09C6ACAEFE4560BF999FE739E88C09, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:31,595 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:31,595 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796291403_0001_1_00_000000_0
2016-01-14 18:31:31,595 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:31,595 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,596 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796291403_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:31,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=16172E1960714427819A13672CAA68B0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:31,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:31,597 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 16172E1960714427819A13672CAA68B0
2016-01-14 18:31:31,597 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 16172E1960714427819A13672CAA68B0
2016-01-14 18:31:31,598 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:31,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DD09C6ACAEFE4560BF999FE739E88C09, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:31,598 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 16172E1960714427819A13672CAA68B0
2016-01-14 18:31:31,598 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:31,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:31,599 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DD09C6ACAEFE4560BF999FE739E88C09
2016-01-14 18:31:31,602 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:31,602 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DD09C6ACAEFE4560BF999FE739E88C09
2016-01-14 18:31:31,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DD09C6ACAEFE4560BF999FE739E88C09
2016-01-14 18:31:31,601 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 16172E1960714427819A13672CAA68B0
2016-01-14 18:31:31,603 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E98F2153CB714A82ADF2F7BCB6C01446, ordinal: 0
2016-01-14 18:31:31,603 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E98F2153CB714A82ADF2F7BCB6C01446, mem on start (mb), free: 255, total: 370, max: 672
2016-01-14 18:31:31,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:31,603 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:31,604 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:31,604 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:31,604 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:31,604 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:31,604 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:31,604 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:31,605 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:31,605 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 16172E1960714427819A13672CAA68B0, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:31,605 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DD09C6ACAEFE4560BF999FE739E88C09, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:31,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:31,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 16172E1960714427819A13672CAA68B0 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:31,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 16172E1960714427819A13672CAA68B0
2016-01-14 18:31:31,606 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:31,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 16172E1960714427819A13672CAA68B0
2016-01-14 18:31:31,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:31,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:31,606 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:31,606 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796291403_0001_1_00_000000_0
2016-01-14 18:31:31,606 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:31,607 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3e29076
2016-01-14 18:31:31,608 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3e29076
2016-01-14 18:31:31,608 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:31,608 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:31,610 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:31,611 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 16172E1960714427819A13672CAA68B0
2016-01-14 18:31:31,611 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 155B609BEFCE42DA98DB1DC3AAA52A37
2016-01-14 18:31:31,611 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 155B609BEFCE42DA98DB1DC3AAA52A37
2016-01-14 18:31:31,612 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@1ea0e836; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@55ed40ef; comparator=cascading.tuple.hadoop.util.TupleComparator@4b4236fb; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:31,612 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:31,612 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:31,612 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:31,625 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:31,625 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:31,625 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:31,625 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:31,625 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:31,626 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:31,626 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:31,627 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E98F2153CB714A82ADF2F7BCB6C01446, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:31,630 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796291403_0001_1_00_000000_0
2016-01-14 18:31:31,630 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E98F2153CB714A82ADF2F7BCB6C01446, mem on close (mb), free: 152, total: 370, max: 672
2016-01-14 18:31:31,630 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:31,630 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:31,630 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:31,630 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:31,632 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:31 Completed running task attempt: attempt_1452796291403_0001_1_00_000000_0
2016-01-14 18:31:31,634 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796291403_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:31,634 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796291403_0001_1_00, eventCount=1
2016-01-14 18:31:31,635 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:31,635 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E98F2153CB714A82ADF2F7BCB6C01446, taskAttemptId=attempt_1452796291403_0001_1_00_000000_0, startTime=1452796291589, finishTime=1452796291635, timeTaken=46, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291627, Process_Duration=3, Process_End_Time=1452796291630, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:31,635 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291403_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:31,635 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:31,635 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796291403_0001_1_00_000000_0
2016-01-14 18:31:31,636 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:31,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:TASK_FINISHED]: vertexName=E98F2153CB714A82ADF2F7BCB6C01446, taskId=task_1452796291403_0001_1_00_000000, startTime=1452796291589, finishTime=1452796291636, timeTaken=47, status=SUCCEEDED, successfulAttemptID=attempt_1452796291403_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291627, Process_Duration=3, Process_End_Time=1452796291630, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:31,636 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291403_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:31,636 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446] : 1
2016-01-14 18:31:31,636 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446]
2016-01-14 18:31:31,637 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:31,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:VERTEX_FINISHED]: vertexName=E98F2153CB714A82ADF2F7BCB6C01446, vertexId=vertex_1452796291403_0001_1_00, initRequestedTime=1452796291564, initedTime=1452796291576, startRequestedTime=1452796291577, startedTime=1452796291577, finishTime=1452796291636, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291627, Process_Duration=3, Process_End_Time=1452796291630, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796291589, firstTasksToStart=[ task_1452796291403_0001_1_00_000000 ], lastTaskFinishTime=1452796291635, lastTasksToFinish=[ task_1452796291403_0001_1_00_000000 ], minTaskDuration=46, maxTaskDuration=46, avgTaskDuration=46.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796291403_0001_1_00_000000 ], longestDurationTasks=[ task_1452796291403_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:31,637 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:31,638 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:31,638 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09] attempt: attempt_1452796291403_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:31,638 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DD09C6ACAEFE4560BF999FE739E88C09
2016-01-14 18:31:31,638 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09]
2016-01-14 18:31:31,638 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796291403_0001_00_000001, stoppedTime=1452796291638, exitStatus=0
2016-01-14 18:31:31,638 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:31,638 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:31,639 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:TASK_STARTED]: vertexName=DD09C6ACAEFE4560BF999FE739E88C09, taskId=task_1452796291403_0001_1_01_000000, scheduledTime=1452796291639, launchTime=1452796291639
2016-01-14 18:31:31,639 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291403_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:31,639 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09]
2016-01-14 18:31:31,639 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796291403_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:31,639 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: DD09C6ACAEFE4560BF999FE739E88C09, VertexParallelism: 1, TaskAttemptID:attempt_1452796291403_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E98F2153CB714A82ADF2F7BCB6C01446, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6EF8925BE6EA4FF5ABEBC00C07B447D4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:31,639 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291403_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:31,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:31,640 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:31,640 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:31,640 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:31,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796291627, Process_Duration=3, Process_End_Time=1452796291630, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:31,641 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:31,641 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:31,641 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796291403_0001_00_000002
2016-01-14 18:31:31,641 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:31,641 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796291403_0001_00_000001
2016-01-14 18:31:31,642 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796291403_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:31,642 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796291403_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:31,642 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796291403_0001_00_000001, Exiting
2016-01-14 18:31:31,642 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796291403_0001_00_000001
2016-01-14 18:31:31,642 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796291403_0001_00_000001
2016-01-14 18:31:31,642 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796291403_0001_00_000002
2016-01-14 18:31:31,643 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796291403_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:31,643 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796291403_0001_00_000001 completed successfully
2016-01-14 18:31:31,643 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:31,643 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:31,645 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:31,645 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:31,646 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796291403_0001_00_000002, launchTime=1452796291645
2016-01-14 18:31:31,646 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796291403_0001_00_000002
2016-01-14 18:31:31,647 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796291403_0001_1_01_000000_0] to container: [container_1452796291403_0001_00_000002]
2016-01-14 18:31:31,647 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:31,647 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796291403_0001_00_000002 given task: attempt_1452796291403_0001_1_01_000000_0
2016-01-14 18:31:31,647 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796291403_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796291403_0001_1_01_000000_0
2016-01-14 18:31:31,647 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796291403_0001_1_01_000000_0] started. Is using containerId: [container_1452796291403_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:31,647 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD09C6ACAEFE4560BF999FE739E88C09, taskAttemptId=attempt_1452796291403_0001_1_01_000000_0, startTime=1452796291647, containerId=container_1452796291403_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796291403_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:31 Starting to run new task attempt: attempt_1452796291403_0001_1_01_000000_0
2016-01-14 18:31:31,647 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796291403_0001_1_01_000000_0
2016-01-14 18:31:31,647 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:31,648 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:31,648 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:31,647 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291403_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:31,648 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291403_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:31,648 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:31,648 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:31,648 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:31,649 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:31,649 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D] every/(1/1) ...latform/0_UNKNOWN_RESULTS, VertexName: DD09C6ACAEFE4560BF999FE739E88C09, VertexParallelism: 1, TaskAttemptID:attempt_1452796291403_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E98F2153CB714A82ADF2F7BCB6C01446, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=6EF8925BE6EA4FF5ABEBC00C07B447D4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:31,649 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:31,649 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796291403_0001_1_01_000000_0
2016-01-14 18:31:31,649 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:31,650 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E98F2153CB714A82ADF2F7BCB6C01446, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:31,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:31,651 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E98F2153CB714A82ADF2F7BCB6C01446
2016-01-14 18:31:31,653 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796291403_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:31,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,654 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:31,654 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:31,654 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E98F2153CB714A82ADF2F7BCB6C01446
2016-01-14 18:31:31,654 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E98F2153CB714A82ADF2F7BCB6C01446
2016-01-14 18:31:31,656 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=6EF8925BE6EA4FF5ABEBC00C07B447D4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:31,657 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:31,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:31,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 6EF8925BE6EA4FF5ABEBC00C07B447D4
2016-01-14 18:31:31,658 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:31,658 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 6EF8925BE6EA4FF5ABEBC00C07B447D4
2016-01-14 18:31:31,663 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD09C6ACAEFE4560BF999FE739E88C09, ordinal: 1
2016-01-14 18:31:31,663 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD09C6ACAEFE4560BF999FE739E88C09, mem on start (mb), free: 146, total: 370, max: 672
2016-01-14 18:31:31,663 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:31,663 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:31,663 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:31,666 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:31,666 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 6EF8925BE6EA4FF5ABEBC00C07B447D4
2016-01-14 18:31:31,666 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:31,666 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:31,666 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:31,666 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:31,666 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:31,666 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:31,667 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:31,667 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E98F2153CB714A82ADF2F7BCB6C01446, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:31,667 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 6EF8925BE6EA4FF5ABEBC00C07B447D4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:31,667 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:31,667 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E98F2153CB714A82ADF2F7BCB6C01446 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:31,667 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E98F2153CB714A82ADF2F7BCB6C01446
2016-01-14 18:31:31,667 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:31,668 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:31,668 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:31,668 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:31,668 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E98F2153CB714A82ADF2F7BCB6C01446 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:31,669 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:31,669 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:31,669 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:31,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E98F2153CB714A82ADF2F7BCB6C01446
2016-01-14 18:31:31,671 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:31,671 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796291403_0001_1_01_000000_0
2016-01-14 18:31:31,673 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796291403_0001_1_00_000000_0_10153, runDuration: 32489, ]
2016-01-14 18:31:31,675 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796291403_0001_1_00_000000_0_10153] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796291675, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:31,675 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E98F2153CB714A82ADF2F7BCB6C01446
2016-01-14 18:31:31,675 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:31,675 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:31,675 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E98F2153CB714A82ADF2F7BCB6C01446] #26 in 1ms
2016-01-14 18:31:31,675 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E98F2153CB714A82ADF2F7BCB6C01446] #26, status:false, isInterrupted:false
2016-01-14 18:31:31,676 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 155B609BEFCE42DA98DB1DC3AAA52A37
2016-01-14 18:31:31,678 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS"], id: 6EF8925BE6EA4FF5ABEBC00C07B447D4
2016-01-14 18:31:31,678 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_RESULTS"]
2016-01-14 18:31:31,678 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 155B609BEFCE42DA98DB1DC3AAA52A37, for 1 inputs
2016-01-14 18:31:31,678 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:31,678 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:31,678 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:31,679 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:31,679 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:31,679 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E98F2153CB714A82ADF2F7BCB6C01446
2016-01-14 18:31:31,679 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD09C6ACAEFE4560BF999FE739E88C09, all 1 inputs ready in: 00:00:00.001
2016-01-14 18:31:31,679 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:31,680 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@37715099; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7062893e
2016-01-14 18:31:31,683 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:31,684 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:31,684 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796291403_0001_1_01_000000_0
2016-01-14 18:31:31,684 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796291403_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:31,685 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796291403_0001_1_01_000000_0
2016-01-14 18:31:31,685 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD09C6ACAEFE4560BF999FE739E88C09, mem on close (mb), free: 142, total: 370, max: 672
2016-01-14 18:31:31,685 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E98F2153CB714A82ADF2F7BCB6C01446
2016-01-14 18:31:31,685 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:31,685 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:31,685 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:31,686 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:31,686 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:31 Completed running task attempt: attempt_1452796291403_0001_1_01_000000_0
2016-01-14 18:31:31,686 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796291403_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:31,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:31,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD09C6ACAEFE4560BF999FE739E88C09, taskAttemptId=attempt_1452796291403_0001_1_01_000000_0, startTime=1452796291647, finishTime=1452796291687, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291679, Process_Duration=6, Process_End_Time=1452796291685, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:31,687 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291403_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:31,687 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:31,687 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796291403_0001_1_01_000000_0
2016-01-14 18:31:31,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:TASK_FINISHED]: vertexName=DD09C6ACAEFE4560BF999FE739E88C09, taskId=task_1452796291403_0001_1_01_000000, startTime=1452796291647, finishTime=1452796291687, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1452796291403_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291679, Process_Duration=6, Process_End_Time=1452796291685, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:31,688 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:31,688 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291403_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:31,688 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09] : 1
2016-01-14 18:31:31,688 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09]
2016-01-14 18:31:31,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796291679, Process_Duration=6, Process_End_Time=1452796291685, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:31,689 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:31,689 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:31,689 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796291403_0001_00_000002
2016-01-14 18:31:31,689 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:31,689 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:VERTEX_FINISHED]: vertexName=DD09C6ACAEFE4560BF999FE739E88C09, vertexId=vertex_1452796291403_0001_1_01, initRequestedTime=1452796291565, initedTime=1452796291572, startRequestedTime=1452796291580, startedTime=1452796291580, finishTime=1452796291688, timeTaken=108, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=388497408, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796291679, Process_Duration=6, Process_End_Time=1452796291685, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796291647, firstTasksToStart=[ task_1452796291403_0001_1_01_000000 ], lastTaskFinishTime=1452796291687, lastTasksToFinish=[ task_1452796291403_0001_1_01_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796291403_0001_1_01_000000 ], longestDurationTasks=[ task_1452796291403_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:31,690 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:31,690 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:31,690 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796291403_0001_00_000002, stoppedTime=1452796291690, exitStatus=0
2016-01-14 18:31:31,690 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:31,690 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:31,691 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796291403_0001_1
2016-01-14 18:31:31,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796291403_0001_1
2016-01-14 18:31:31,691 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 6EF8925BE6EA4FF5ABEBC00C07B447D4 for vertex: vertex_1452796291403_0001_1_01 [DD09C6ACAEFE4560BF999FE739E88C09]
2016-01-14 18:31:31,691 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:31,696 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796291403_0001_1_00 [E98F2153CB714A82ADF2F7BCB6C01446]
2016-01-14 18:31:31,698 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796291403_0001_1, queueSize=0
2016-01-14 18:31:31,699 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:31,700 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291403_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796291403_0001_1, startTime=1452796291563, finishTime=1452796291696, timeTaken=133, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=776994816, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592583306, Process_Duration=9, Process_End_Time=2905592583315, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:31,700 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796291403_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:31,700 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796291403_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:31,700 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796291403_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:31,701 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796291403_0001_00_000002: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:31,701 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796291403_0001_00_000002, Exiting
2016-01-14 18:31:31,701 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796291403_0001_00_000002
2016-01-14 18:31:31,701 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796291403_0001_00_000002
2016-01-14 18:31:31,702 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796291403_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:31,702 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796291403_0001_00_000002 completed successfully
2016-01-14 18:31:31,702 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796291403_0001_00_000002
2016-01-14 18:31:31 Completed Dag: dag_1452796291403_0001_1
2016-01-14 18:31:31,703 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796291403_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:31,703 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796291403_0001_1_post
2016-01-14 18:31:31,703 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:31,703 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:31,703 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291403_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:31,707 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_UNKNOWN_RESULTS, applicationId=application_1452796291403_0001
2016-01-14 18:31:31,709 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:31,709 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:31,709 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:31,709 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:31,709 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:31,709 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:31,710 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:31,710 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:31,710 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:31,710 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:31,711 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:31,711 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:31,711 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:31,717 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8203646412D24F5F8220755CB11CF580/E93B4FA4C73A45FDB78A4EA88E41F62D/.tez/application_1452796291403_0001
2016-01-14 18:31:31,717 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.331
2016-01-14 18:31:31,786 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_RESULTS
2016-01-14 18:31:31,806 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:31,807 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:31,847 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 18:31:31,852 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 18:31:31,853 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:31,853 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:31,854 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:31,868 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:31,873 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL
2016-01-14 18:31:31,920 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:31,920 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:31,920 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL"]
2016-01-14 18:31:31,920 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:31,920 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:31,920 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:31,922 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/0_UNKNOWN_ALL
2016-01-14 18:31:31,922 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:31,936 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:31,936 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:31,943 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:31,947 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:31,948 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:31,948 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:31,957 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD/.tez/application_1452796291947_0001 doesn't exist and is created
2016-01-14 18:31:31,972 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:31,972 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:31,973 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD/.tez/application_1452796291947_0001
2016-01-14 18:31:31,974 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796291947_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:31,984 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:31,985 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:31,986 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:31,987 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:31,987 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD/.tez/application_1452796291947_0001/localmode-log-dir
2016-01-14 18:31:31,987 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD/.tez/application_1452796291947_0001/localmode-log-dir/history.txt.appattempt_1452796291947_0001_000000, maxErrors=10
2016-01-14 18:31:31,988 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:31,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796291947_0001_000000, appSubmitTime=1452796291974, launchTime=1452796291974
2016-01-14 18:31:31,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:31,993 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:33336
2016-01-14 18:31:31,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:31,997 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:31,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:32,000 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:32,001 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796291947_0001_000000, startTime=1452796292001
2016-01-14 18:31:32,001 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:32,001 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:32,072 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:32,072 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:32,072 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1452796291947_0001, dagName=[D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-01-14 18:31:32,088 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-01-14 18:31:32,089 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796291947_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD/.tez/application_1452796291947_0001/localmode-log-dir/dag_1452796291947_0001_1.dot
2016-01-14 18:31:32,089 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD/.tez/application_1452796291947_0001/localmode-log-dir/dag_1452796291947_0001_1-tez-dag.pb.txt
2016-01-14 18:31:32,095 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796291947_0001_1
2016-01-14 18:31:32,096 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:32,096 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-01-14 18:31:32 Running Dag: dag_1452796291947_0001_1
2016-01-14 18:31:32,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796291947_0001_1, submitTime=1452796292088
2016-01-14 18:31:32,105 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:32,106 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DD199DC30D1D48F7B8618EBC33D64247
2016-01-14 18:31:32,106 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:32,106 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796291947_0001_1, initTime=1452796292105
2016-01-14 18:31:32,107 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796291947_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:32,106 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:32,107 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:DAG_STARTED]: dagID=dag_1452796291947_0001_1, startTime=1452796292107
2016-01-14 18:31:32,107 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:32,107 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:32,107 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1452796291947_0001, dagName=[D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD] every/(1/1) ...blyplatform/0_UNKNOWN_ALL
2016-01-14 18:31:32,107 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796291947_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:32,108 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A0B462E3F7384F68BE16BFC9CC41ED9B : {319D74B9A1C04214AA8E4B479EB369EC={InputName=319D74B9A1C04214AA8E4B479EB369EC}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:32,108 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 319D74B9A1C04214AA8E4B479EB369EC, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:32,108 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B]
2016-01-14 18:31:32,108 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B]
2016-01-14 18:31:32,108 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B]
2016-01-14 18:31:32,108 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B]. Starting root input initializers: 1
2016-01-14 18:31:32,109 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:32,109 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247]
2016-01-14 18:31:32,109 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 319D74B9A1C04214AA8E4B479EB369EC on vertex vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B]
2016-01-14 18:31:32,111 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:32,111 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 319D74B9A1C04214AA8E4B479EB369EC asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:32,111 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:32,112 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:32,112 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:32,113 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247]
2016-01-14 18:31:32,113 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247]
2016-01-14 18:31:32,113 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247]
2016-01-14 18:31:32,113 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3AF9EB1DDAD8467E9BD5D97C0A13436F, vertexId=vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:32,113 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3AF9EB1DDAD8467E9BD5D97C0A13436F, vertexId=vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247]
2016-01-14 18:31:32,113 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:32,114 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 319D74B9A1C04214AA8E4B479EB369EC on vertex vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B]
2016-01-14 18:31:32,115 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3AF9EB1DDAD8467E9BD5D97C0A13436F, vertexName=DD199DC30D1D48F7B8618EBC33D64247, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:32,116 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3AF9EB1DDAD8467E9BD5D97C0A13436F, vertexId=vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247]
2016-01-14 18:31:32,117 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:32,117 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DD199DC30D1D48F7B8618EBC33D64247, vertexId=vertex_1452796291947_0001_1_01, initRequestedTime=1452796292109, initedTime=1452796292116, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:32,117 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:32,117 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:32,117 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {319D74B9A1C04214AA8E4B479EB369EC=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:32,118 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B] parallelism set to 1
2016-01-14 18:31:32,118 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:32,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796291947_0001_1_00, eventCount=1
2016-01-14 18:31:32,118 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:32,118 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A0B462E3F7384F68BE16BFC9CC41ED9B, vertexId=vertex_1452796291947_0001_1_00, initRequestedTime=1452796292108, initedTime=1452796292118, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:32,119 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B] with distanceFromRoot: 0
2016-01-14 18:31:32,119 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:32,119 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A0B462E3F7384F68BE16BFC9CC41ED9B
2016-01-14 18:31:32,119 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B]
2016-01-14 18:31:32,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:32,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796291947_0001_1_00, startRequestedTime=1452796292119, startedTime=1452796292119
2016-01-14 18:31:32,120 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:32,120 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:32,120 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:TASK_STARTED]: vertexName=A0B462E3F7384F68BE16BFC9CC41ED9B, taskId=task_1452796291947_0001_1_00_000000, scheduledTime=1452796292120, launchTime=1452796292120
2016-01-14 18:31:32,121 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291947_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:32,121 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796291947_0001_1_00 for vertex: vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247] numStartedSources: 1 numSources: 1
2016-01-14 18:31:32,121 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247] with distanceFromRoot: 1
2016-01-14 18:31:32,121 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B]
2016-01-14 18:31:32,122 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A0B462E3F7384F68BE16BFC9CC41ED9B in vertex: DD199DC30D1D48F7B8618EBC33D64247
2016-01-14 18:31:32,122 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DD199DC30D1D48F7B8618EBC33D64247 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:32,122 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:32,122 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796291947_0001_1_01, startRequestedTime=1452796292121, startedTime=1452796292121
2016-01-14 18:31:32,122 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:32,123 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: A0B462E3F7384F68BE16BFC9CC41ED9B, VertexParallelism: 1, TaskAttemptID:attempt_1452796291947_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=319D74B9A1C04214AA8E4B479EB369EC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD199DC30D1D48F7B8618EBC33D64247, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:32,123 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291947_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:32,123 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:32,124 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:32,124 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:32,124 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:32,125 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796291947_0001_00_000001
2016-01-14 18:31:32,125 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:32,126 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796291947_0001_00_000001
2016-01-14 18:31:32,126 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:32,127 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:32,127 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:32,127 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796291947_0001_00_000001, launchTime=1452796292127
2016-01-14 18:31:32,128 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796291947_0001_00_000001
2016-01-14 18:31:32,128 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796291947_0001, with dag id: dag_1452796291947_0001_1
2016-01-14 18:31:32,128 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796291947_0001_1_00_000000_0] to container: [container_1452796291947_0001_00_000001]
2016-01-14 18:31:32,129 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:32,129 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796291947_0001_1_00_000000_0] started. Is using containerId: [container_1452796291947_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:32,129 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A0B462E3F7384F68BE16BFC9CC41ED9B, taskAttemptId=attempt_1452796291947_0001_1_00_000000_0, startTime=1452796292129, containerId=container_1452796291947_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796291947_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:32,130 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291947_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:32,130 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:32,130 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291947_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:32,130 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796291947_0001_00_000001 given task: attempt_1452796291947_0001_1_00_000000_0
2016-01-14 18:31:32,130 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796291947_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796291947_0001_1_00_000000_0
2016-01-14 18:31:32 Starting to run new task attempt: attempt_1452796291947_0001_1_00_000000_0
2016-01-14 18:31:32,131 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796291947_0001_1_00_000000_0
2016-01-14 18:31:32,131 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:32,131 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:32,131 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:32,131 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:32,131 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:32,131 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:32,131 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: A0B462E3F7384F68BE16BFC9CC41ED9B, VertexParallelism: 1, TaskAttemptID:attempt_1452796291947_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=319D74B9A1C04214AA8E4B479EB369EC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DD199DC30D1D48F7B8618EBC33D64247, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:32,132 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:32,132 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796291947_0001_1_00_000000_0
2016-01-14 18:31:32,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:32,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,133 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796291947_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:32,134 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DD199DC30D1D48F7B8618EBC33D64247, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:32,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=319D74B9A1C04214AA8E4B479EB369EC, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:32,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:32,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 319D74B9A1C04214AA8E4B479EB369EC
2016-01-14 18:31:32,136 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 319D74B9A1C04214AA8E4B479EB369EC
2016-01-14 18:31:32,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:32,137 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DD199DC30D1D48F7B8618EBC33D64247
2016-01-14 18:31:32,137 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:32,137 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:32,137 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 319D74B9A1C04214AA8E4B479EB369EC
2016-01-14 18:31:32,138 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 319D74B9A1C04214AA8E4B479EB369EC
2016-01-14 18:31:32,140 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:32,140 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DD199DC30D1D48F7B8618EBC33D64247
2016-01-14 18:31:32,140 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DD199DC30D1D48F7B8618EBC33D64247
2016-01-14 18:31:32,143 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A0B462E3F7384F68BE16BFC9CC41ED9B, ordinal: 0
2016-01-14 18:31:32,143 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A0B462E3F7384F68BE16BFC9CC41ED9B, mem on start (mb), free: 248, total: 372, max: 672
2016-01-14 18:31:32,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,143 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:32,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:32,144 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:32,144 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:32,144 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:32,144 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:32,144 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:32,145 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:32,145 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:32,145 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 319D74B9A1C04214AA8E4B479EB369EC, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:32,145 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DD199DC30D1D48F7B8618EBC33D64247, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:32,145 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:32,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 319D74B9A1C04214AA8E4B479EB369EC being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:32,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:32,146 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 319D74B9A1C04214AA8E4B479EB369EC
2016-01-14 18:31:32,146 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:32,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 319D74B9A1C04214AA8E4B479EB369EC
2016-01-14 18:31:32,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:32,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:32,147 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796291947_0001_1_00_000000_0
2016-01-14 18:31:32,148 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:32,148 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@6abdc70f
2016-01-14 18:31:32,149 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@6abdc70f
2016-01-14 18:31:32,149 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:32,149 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:32,151 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:32,151 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 319D74B9A1C04214AA8E4B479EB369EC
2016-01-14 18:31:32,152 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 1670FB9A1502434086E711D4BC6AB2DD
2016-01-14 18:31:32,152 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 1670FB9A1502434086E711D4BC6AB2DD
2016-01-14 18:31:32,152 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@cf6275d; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@74925150; comparator=cascading.tuple.hadoop.util.TupleComparator@321896fa; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:32,153 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:32,153 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:32,153 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:32,169 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:32,169 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:32,169 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:32,169 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:32,169 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:32,170 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:32,170 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:32,171 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A0B462E3F7384F68BE16BFC9CC41ED9B, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:32,174 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796291947_0001_1_00_000000_0
2016-01-14 18:31:32,174 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A0B462E3F7384F68BE16BFC9CC41ED9B, mem on close (mb), free: 146, total: 372, max: 672
2016-01-14 18:31:32,174 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:32,174 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:32,175 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:32,175 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:32,176 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:32 Completed running task attempt: attempt_1452796291947_0001_1_00_000000_0
2016-01-14 18:31:32,178 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796291947_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:32,178 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796292171, Process_Duration=3, Process_End_Time=1452796292174, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:32,178 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796291947_0001_1_00, eventCount=1
2016-01-14 18:31:32,178 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:32,178 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:32,179 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796291947_0001_00_000001
2016-01-14 18:31:32,179 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:32,179 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A0B462E3F7384F68BE16BFC9CC41ED9B, taskAttemptId=attempt_1452796291947_0001_1_00_000000_0, startTime=1452796292129, finishTime=1452796292179, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796292171, Process_Duration=3, Process_End_Time=1452796292174, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:32,179 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291947_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:32,179 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:32,179 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796291947_0001_1_00_000000_0
2016-01-14 18:31:32,180 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:32,180 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:TASK_FINISHED]: vertexName=A0B462E3F7384F68BE16BFC9CC41ED9B, taskId=task_1452796291947_0001_1_00_000000, startTime=1452796292129, finishTime=1452796292179, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1452796291947_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796292171, Process_Duration=3, Process_End_Time=1452796292174, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:32,180 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291947_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:32,180 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B] : 1
2016-01-14 18:31:32,180 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B]
2016-01-14 18:31:32,181 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:32,181 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:VERTEX_FINISHED]: vertexName=A0B462E3F7384F68BE16BFC9CC41ED9B, vertexId=vertex_1452796291947_0001_1_00, initRequestedTime=1452796292108, initedTime=1452796292118, startRequestedTime=1452796292119, startedTime=1452796292119, finishTime=1452796292180, timeTaken=61, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796292171, Process_Duration=3, Process_End_Time=1452796292174, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796292129, firstTasksToStart=[ task_1452796291947_0001_1_00_000000 ], lastTaskFinishTime=1452796292179, lastTasksToFinish=[ task_1452796291947_0001_1_00_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796291947_0001_1_00_000000 ], longestDurationTasks=[ task_1452796291947_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:32,181 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:32,181 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:32,181 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247] attempt: attempt_1452796291947_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:32,181 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DD199DC30D1D48F7B8618EBC33D64247
2016-01-14 18:31:32,181 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247]
2016-01-14 18:31:32,181 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:32,182 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796291947_0001_00_000001, stoppedTime=1452796292182, exitStatus=0
2016-01-14 18:31:32,182 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:32,182 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:32,184 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:TASK_STARTED]: vertexName=DD199DC30D1D48F7B8618EBC33D64247, taskId=task_1452796291947_0001_1_01_000000, scheduledTime=1452796292184, launchTime=1452796292184
2016-01-14 18:31:32,184 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:32,184 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291947_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:32,184 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796291947_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:32,184 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247]
2016-01-14 18:31:32,185 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:32,186 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: DD199DC30D1D48F7B8618EBC33D64247, VertexParallelism: 1, TaskAttemptID:attempt_1452796291947_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A0B462E3F7384F68BE16BFC9CC41ED9B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3AF9EB1DDAD8467E9BD5D97C0A13436F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:32,186 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291947_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:32,186 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:32,186 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796291947_0001_00_000002
2016-01-14 18:31:32,187 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:32,188 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796291947_0001_00_000002
2016-01-14 18:31:32,188 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:32,188 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:32,189 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:32,189 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796291947_0001_00_000002, launchTime=1452796292189
2016-01-14 18:31:32,189 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796291947_0001_00_000002
2016-01-14 18:31:32,189 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796291947_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:32,190 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796291947_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:32,190 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796291947_0001_00_000001, Exiting
2016-01-14 18:31:32,190 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796291947_0001_00_000001
2016-01-14 18:31:32,190 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796291947_0001_00_000001
2016-01-14 18:31:32,190 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796291947_0001_1_01_000000_0] to container: [container_1452796291947_0001_00_000002]
2016-01-14 18:31:32,190 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:32,191 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796291947_0001_1_01_000000_0] started. Is using containerId: [container_1452796291947_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:32,191 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DD199DC30D1D48F7B8618EBC33D64247, taskAttemptId=attempt_1452796291947_0001_1_01_000000_0, startTime=1452796292190, containerId=container_1452796291947_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796291947_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:32,191 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291947_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:32,191 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:32,191 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796291947_0001_00_000002 given task: attempt_1452796291947_0001_1_01_000000_0
2016-01-14 18:31:32,191 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291947_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:32,191 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796291947_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796291947_0001_1_01_000000_0
2016-01-14 18:31:32 Starting to run new task attempt: attempt_1452796291947_0001_1_01_000000_0
2016-01-14 18:31:32,192 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796291947_0001_1_01_000000_0
2016-01-14 18:31:32,192 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:32,192 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:32,192 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:32,192 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:32,192 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:32,192 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:32,192 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD] every/(1/1) ...blyplatform/0_UNKNOWN_ALL, VertexName: DD199DC30D1D48F7B8618EBC33D64247, VertexParallelism: 1, TaskAttemptID:attempt_1452796291947_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A0B462E3F7384F68BE16BFC9CC41ED9B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3AF9EB1DDAD8467E9BD5D97C0A13436F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:32,192 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:32,192 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796291947_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:32,192 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796291947_0001_00_000001 completed successfully
2016-01-14 18:31:32,192 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:32,193 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796291947_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:32,194 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796291947_0001_1_01_000000_0
2016-01-14 18:31:32,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:32,194 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A0B462E3F7384F68BE16BFC9CC41ED9B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:32,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3AF9EB1DDAD8467E9BD5D97C0A13436F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:32,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:32,196 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3AF9EB1DDAD8467E9BD5D97C0A13436F
2016-01-14 18:31:32,196 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:32,196 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3AF9EB1DDAD8467E9BD5D97C0A13436F
2016-01-14 18:31:32,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:32,197 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A0B462E3F7384F68BE16BFC9CC41ED9B
2016-01-14 18:31:32,198 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:32,200 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:32,200 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:32,200 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A0B462E3F7384F68BE16BFC9CC41ED9B
2016-01-14 18:31:32,201 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A0B462E3F7384F68BE16BFC9CC41ED9B
2016-01-14 18:31:32,204 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DD199DC30D1D48F7B8618EBC33D64247, ordinal: 1
2016-01-14 18:31:32,204 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD199DC30D1D48F7B8618EBC33D64247, mem on start (mb), free: 140, total: 372, max: 672
2016-01-14 18:31:32,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:32,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:32,208 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:32,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3AF9EB1DDAD8467E9BD5D97C0A13436F
2016-01-14 18:31:32,208 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:32,208 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:32,208 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:32,208 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:32,208 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:32,209 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:32,209 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:32,209 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3AF9EB1DDAD8467E9BD5D97C0A13436F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:32,209 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A0B462E3F7384F68BE16BFC9CC41ED9B, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:32,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:32,209 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A0B462E3F7384F68BE16BFC9CC41ED9B
2016-01-14 18:31:32,210 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:32,210 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:32,210 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A0B462E3F7384F68BE16BFC9CC41ED9B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:32,210 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:32,211 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:32,211 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:32,210 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A0B462E3F7384F68BE16BFC9CC41ED9B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:32,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:32,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:32,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A0B462E3F7384F68BE16BFC9CC41ED9B
2016-01-14 18:31:32,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:32,216 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796291947_0001_1_01_000000_0
2016-01-14 18:31:32,216 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796291947_0001_1_00_000000_0_10159, runDuration: 39677, ]
2016-01-14 18:31:32,219 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796291947_0001_1_00_000000_0_10159] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796292219, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:32,219 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A0B462E3F7384F68BE16BFC9CC41ED9B
2016-01-14 18:31:32,220 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:32,220 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:32,220 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 1670FB9A1502434086E711D4BC6AB2DD
2016-01-14 18:31:32,221 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL"], id: 3AF9EB1DDAD8467E9BD5D97C0A13436F
2016-01-14 18:31:32,221 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_ALL"]
2016-01-14 18:31:32,221 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 1670FB9A1502434086E711D4BC6AB2DD, for 1 inputs
2016-01-14 18:31:32,221 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A0B462E3F7384F68BE16BFC9CC41ED9B] #27 in 3ms
2016-01-14 18:31:32,222 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A0B462E3F7384F68BE16BFC9CC41ED9B] #27, status:false, isInterrupted:false
2016-01-14 18:31:32,223 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:32,223 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:32,223 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:32,223 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:32,223 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:32,223 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A0B462E3F7384F68BE16BFC9CC41ED9B
2016-01-14 18:31:32,223 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:32,223 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DD199DC30D1D48F7B8618EBC33D64247, all 1 inputs ready in: 00:00:00.002
2016-01-14 18:31:32,226 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@25264374; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@731687b9
2016-01-14 18:31:32,230 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:32,230 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:32,231 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796291947_0001_1_01_000000_0
2016-01-14 18:31:32,231 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796291947_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:32,232 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796291947_0001_1_01_000000_0
2016-01-14 18:31:32,232 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DD199DC30D1D48F7B8618EBC33D64247, mem on close (mb), free: 136, total: 372, max: 672
2016-01-14 18:31:32,232 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A0B462E3F7384F68BE16BFC9CC41ED9B
2016-01-14 18:31:32,236 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:32,236 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:32,236 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:32,236 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:32,236 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:32 Completed running task attempt: attempt_1452796291947_0001_1_01_000000_0
2016-01-14 18:31:32,237 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796291947_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:32,237 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:32,237 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DD199DC30D1D48F7B8618EBC33D64247, taskAttemptId=attempt_1452796291947_0001_1_01_000000_0, startTime=1452796292190, finishTime=1452796292237, timeTaken=47, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796292224, Process_Duration=8, Process_End_Time=1452796292232, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:32,237 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796291947_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:32,238 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796291947_0001_1_01_000000_0
2016-01-14 18:31:32,238 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:32,238 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:32,238 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:TASK_FINISHED]: vertexName=DD199DC30D1D48F7B8618EBC33D64247, taskId=task_1452796291947_0001_1_01_000000, startTime=1452796292190, finishTime=1452796292238, timeTaken=48, status=SUCCEEDED, successfulAttemptID=attempt_1452796291947_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796292224, Process_Duration=8, Process_End_Time=1452796292232, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:32,238 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796291947_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:32,238 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247] : 1
2016-01-14 18:31:32,238 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247]
2016-01-14 18:31:32,238 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796292224, Process_Duration=8, Process_End_Time=1452796292232, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:32,239 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:32,239 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:32,239 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796291947_0001_00_000002
2016-01-14 18:31:32,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:32,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:VERTEX_FINISHED]: vertexName=DD199DC30D1D48F7B8618EBC33D64247, vertexId=vertex_1452796291947_0001_1_01, initRequestedTime=1452796292109, initedTime=1452796292116, startRequestedTime=1452796292121, startedTime=1452796292121, finishTime=1452796292238, timeTaken=117, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796292224, Process_Duration=8, Process_End_Time=1452796292232, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796292190, firstTasksToStart=[ task_1452796291947_0001_1_01_000000 ], lastTaskFinishTime=1452796292237, lastTasksToFinish=[ task_1452796291947_0001_1_01_000000 ], minTaskDuration=47, maxTaskDuration=47, avgTaskDuration=47.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796291947_0001_1_01_000000 ], longestDurationTasks=[ task_1452796291947_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:32,239 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:32,239 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:32,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796291947_0001_00_000002, stoppedTime=1452796292239, exitStatus=0
2016-01-14 18:31:32,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:32,240 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:32,240 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:32,240 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796291947_0001_1
2016-01-14 18:31:32,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796291947_0001_1
2016-01-14 18:31:32,240 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 3AF9EB1DDAD8467E9BD5D97C0A13436F for vertex: vertex_1452796291947_0001_1_01 [DD199DC30D1D48F7B8618EBC33D64247]
2016-01-14 18:31:32,246 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796291947_0001_1_00 [A0B462E3F7384F68BE16BFC9CC41ED9B]
2016-01-14 18:31:32,248 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796291947_0001_1, queueSize=0
2016-01-14 18:31:32,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:32,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796291947_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796291947_0001_1, startTime=1452796292107, finishTime=1452796292246, timeTaken=139, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=571, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=780140544, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=13, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592584395, Process_Duration=11, Process_End_Time=2905592584406, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:32,249 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796291947_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:32,249 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796291947_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:32,249 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796291947_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:32,249 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796291947_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:32,249 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796291947_0001_00_000002, Exiting
2016-01-14 18:31:32,250 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796291947_0001_00_000002
2016-01-14 18:31:32,250 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796291947_0001_00_000002
2016-01-14 18:31:32,250 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796291947_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:32,250 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796291947_0001_00_000002 completed successfully
2016-01-14 18:31:32,251 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796291947_0001_00_000002
2016-01-14 18:31:32 Completed Dag: dag_1452796291947_0001_1
2016-01-14 18:31:32,251 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796291947_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:32,251 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796291947_0001_1_post
2016-01-14 18:31:32,252 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:32,252 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:32,252 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796291947_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:32,256 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_UNKNOWN_ALL, applicationId=application_1452796291947_0001
2016-01-14 18:31:32,258 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:32,258 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:32,258 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:32,258 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:32,258 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:32,258 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:32,259 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:32,259 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:32,259 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:32,260 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:32,260 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:32,260 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:32,260 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:32,266 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D78C6C2583F944F8B197FE15AF128701/B7978614C1EF4793894018CB751066DD/.tez/application_1452796291947_0001
2016-01-14 18:31:32,266 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.347
2016-01-14 18:31:32,334 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_ALL
2016-01-14 18:31:32,362 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:32,362 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:32,394 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 18:31:32,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 18:31:32,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:32,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:32,398 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:32,412 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:32,416 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE
2016-01-14 18:31:32,460 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:32,460 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:32,461 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE"]
2016-01-14 18:31:32,461 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:32,461 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:32,461 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:32,462 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 18:31:32,463 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:32,475 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:32,480 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:32,480 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:32,480 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:32,489 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4/.tez/application_1452796292480_0001 doesn't exist and is created
2016-01-14 18:31:32,503 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:32,503 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:32,503 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4/.tez/application_1452796292480_0001
2016-01-14 18:31:32,505 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796292480_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:32,515 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:32,516 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:32,517 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:32,517 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:32,517 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4/.tez/application_1452796292480_0001/localmode-log-dir
2016-01-14 18:31:32,518 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4/.tez/application_1452796292480_0001/localmode-log-dir/history.txt.appattempt_1452796292480_0001_000000, maxErrors=10
2016-01-14 18:31:32,518 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:32,518 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796292480_0001_000000, appSubmitTime=1452796292505, launchTime=1452796292505
2016-01-14 18:31:32,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:32,523 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:32,523 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:32,524 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:32,526 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:41504
2016-01-14 18:31:32,527 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:32,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796292480_0001_000000, startTime=1452796292528
2016-01-14 18:31:32,528 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:32,528 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:32,584 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:32,584 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:32,603 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:32,603 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:32,604 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1452796292480_0001, dagName=[C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 18:31:32,619 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 18:31:32,620 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796292480_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4/.tez/application_1452796292480_0001/localmode-log-dir/dag_1452796292480_0001_1.dot
2016-01-14 18:31:32,620 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4/.tez/application_1452796292480_0001/localmode-log-dir/dag_1452796292480_0001_1-tez-dag.pb.txt
2016-01-14 18:31:32,626 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796292480_0001_1
2016-01-14 18:31:32,627 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:32,627 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 18:31:32 Running Dag: dag_1452796292480_0001_1
2016-01-14 18:31:32,635 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:32,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796292480_0001_1, submitTime=1452796292619
2016-01-14 18:31:32,636 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A6C46F0CEBD844E1B0CC85C7FAB4F0AE
2016-01-14 18:31:32,636 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:32,637 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:32,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796292480_0001_1, initTime=1452796292636
2016-01-14 18:31:32,637 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796292480_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:32,637 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:DAG_STARTED]: dagID=dag_1452796292480_0001_1, startTime=1452796292637
2016-01-14 18:31:32,637 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:32,638 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796292480_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:32,638 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B34CC35AAD5145BFAC48D3E29E7EC246 : {BFFE4491AD0C44AF866E088BA6019140={InputName=BFFE4491AD0C44AF866E088BA6019140}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:32,638 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: BFFE4491AD0C44AF866E088BA6019140, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:32,638 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246]
2016-01-14 18:31:32,638 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246]
2016-01-14 18:31:32,638 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246]
2016-01-14 18:31:32,638 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246]. Starting root input initializers: 1
2016-01-14 18:31:32,639 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:32,639 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:32,639 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE]
2016-01-14 18:31:32,639 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1452796292480_0001, dagName=[C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4] every/(1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 18:31:32,641 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: BFFE4491AD0C44AF866E088BA6019140 on vertex vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246]
2016-01-14 18:31:32,642 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:32,642 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE]
2016-01-14 18:31:32,643 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE]
2016-01-14 18:31:32,643 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:32,643 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE]
2016-01-14 18:31:32,643 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input BFFE4491AD0C44AF866E088BA6019140 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:32,643 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7CA0F97A6C874958B01EB1D50A299D0C, vertexId=vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:32,643 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:32,643 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7CA0F97A6C874958B01EB1D50A299D0C, vertexId=vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE]
2016-01-14 18:31:32,644 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:32,645 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:32,645 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: BFFE4491AD0C44AF866E088BA6019140 on vertex vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246]
2016-01-14 18:31:32,646 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7CA0F97A6C874958B01EB1D50A299D0C, vertexName=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:32,646 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7CA0F97A6C874958B01EB1D50A299D0C, vertexId=vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE]
2016-01-14 18:31:32,647 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:32,647 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, vertexId=vertex_1452796292480_0001_1_01, initRequestedTime=1452796292639, initedTime=1452796292647, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:32,647 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:32,647 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:32,647 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {BFFE4491AD0C44AF866E088BA6019140=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:32,648 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246] parallelism set to 1
2016-01-14 18:31:32,648 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:32,648 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796292480_0001_1_00, eventCount=1
2016-01-14 18:31:32,649 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B34CC35AAD5145BFAC48D3E29E7EC246, vertexId=vertex_1452796292480_0001_1_00, initRequestedTime=1452796292638, initedTime=1452796292649, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:32,649 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:32,649 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246] with distanceFromRoot: 0
2016-01-14 18:31:32,649 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:32,649 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B34CC35AAD5145BFAC48D3E29E7EC246
2016-01-14 18:31:32,649 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246]
2016-01-14 18:31:32,649 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:32,650 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796292480_0001_1_00, startRequestedTime=1452796292649, startedTime=1452796292649
2016-01-14 18:31:32,650 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:32,650 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:32,650 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:TASK_STARTED]: vertexName=B34CC35AAD5145BFAC48D3E29E7EC246, taskId=task_1452796292480_0001_1_00_000000, scheduledTime=1452796292650, launchTime=1452796292650
2016-01-14 18:31:32,651 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292480_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:32,651 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796292480_0001_1_00 for vertex: vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] numStartedSources: 1 numSources: 1
2016-01-14 18:31:32,652 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] with distanceFromRoot: 1
2016-01-14 18:31:32,652 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246]
2016-01-14 18:31:32,652 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B34CC35AAD5145BFAC48D3E29E7EC246 in vertex: A6C46F0CEBD844E1B0CC85C7FAB4F0AE
2016-01-14 18:31:32,652 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A6C46F0CEBD844E1B0CC85C7FAB4F0AE with 1 source tasks and 1 pending tasks
2016-01-14 18:31:32,652 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:32,653 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796292480_0001_1_01, startRequestedTime=1452796292652, startedTime=1452796292652
2016-01-14 18:31:32,653 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:32,654 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: B34CC35AAD5145BFAC48D3E29E7EC246, VertexParallelism: 1, TaskAttemptID:attempt_1452796292480_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BFFE4491AD0C44AF866E088BA6019140, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:32,654 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292480_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:32,654 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:32,655 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:32,655 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:32,655 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:32,656 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796292480_0001_00_000001
2016-01-14 18:31:32,656 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:32,657 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796292480_0001, with dag id: dag_1452796292480_0001_1
2016-01-14 18:31:32,657 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796292480_0001_00_000001
2016-01-14 18:31:32,657 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:32,658 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:32,659 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796292480_0001_00_000001, launchTime=1452796292658
2016-01-14 18:31:32,659 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:32,660 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796292480_0001_00_000001
2016-01-14 18:31:32,661 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796292480_0001_1_00_000000_0] to container: [container_1452796292480_0001_00_000001]
2016-01-14 18:31:32,661 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:32,661 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796292480_0001_1_00_000000_0] started. Is using containerId: [container_1452796292480_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:32,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B34CC35AAD5145BFAC48D3E29E7EC246, taskAttemptId=attempt_1452796292480_0001_1_00_000000_0, startTime=1452796292661, containerId=container_1452796292480_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796292480_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:32,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:32,662 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292480_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:32,662 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292480_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:32,662 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796292480_0001_00_000001 given task: attempt_1452796292480_0001_1_00_000000_0
2016-01-14 18:31:32,662 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796292480_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796292480_0001_1_00_000000_0
2016-01-14 18:31:32 Starting to run new task attempt: attempt_1452796292480_0001_1_00_000000_0
2016-01-14 18:31:32,663 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796292480_0001_1_00_000000_0
2016-01-14 18:31:32,663 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:32,663 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:32,663 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:32,663 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:32,664 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:32,664 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:32,664 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: B34CC35AAD5145BFAC48D3E29E7EC246, VertexParallelism: 1, TaskAttemptID:attempt_1452796292480_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BFFE4491AD0C44AF866E088BA6019140, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:32,664 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:32,665 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796292480_0001_1_00_000000_0
2016-01-14 18:31:32,665 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:32,665 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,665 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796292480_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:32,666 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,666 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BFFE4491AD0C44AF866E088BA6019140, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:32,666 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:32,666 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:32,666 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BFFE4491AD0C44AF866E088BA6019140
2016-01-14 18:31:32,667 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: BFFE4491AD0C44AF866E088BA6019140
2016-01-14 18:31:32,667 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:32,667 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A6C46F0CEBD844E1B0CC85C7FAB4F0AE
2016-01-14 18:31:32,669 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:32,669 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:32,669 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: BFFE4491AD0C44AF866E088BA6019140
2016-01-14 18:31:32,669 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BFFE4491AD0C44AF866E088BA6019140
2016-01-14 18:31:32,670 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:32,670 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A6C46F0CEBD844E1B0CC85C7FAB4F0AE
2016-01-14 18:31:32,670 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A6C46F0CEBD844E1B0CC85C7FAB4F0AE
2016-01-14 18:31:32,673 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B34CC35AAD5145BFAC48D3E29E7EC246, ordinal: 0
2016-01-14 18:31:32,673 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B34CC35AAD5145BFAC48D3E29E7EC246, mem on start (mb), free: 240, total: 372, max: 672
2016-01-14 18:31:32,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:32,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:32,673 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:32,673 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:32,673 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:32,674 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:32,674 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:32,674 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:32,674 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:32,674 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BFFE4491AD0C44AF866E088BA6019140, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:32,675 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A6C46F0CEBD844E1B0CC85C7FAB4F0AE, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:32,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:32,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BFFE4491AD0C44AF866E088BA6019140 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:32,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:32,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:32,675 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BFFE4491AD0C44AF866E088BA6019140
2016-01-14 18:31:32,676 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:32,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BFFE4491AD0C44AF866E088BA6019140
2016-01-14 18:31:32,676 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:32,676 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796292480_0001_1_00_000000_0
2016-01-14 18:31:32,677 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:32,677 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1c215a60
2016-01-14 18:31:32,678 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1c215a60
2016-01-14 18:31:32,678 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:32,678 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:32,679 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:32,680 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: BFFE4491AD0C44AF866E088BA6019140
2016-01-14 18:31:32,680 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 75983417F7F14E4D9D8D21B24F21ECDE
2016-01-14 18:31:32,680 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 75983417F7F14E4D9D8D21B24F21ECDE
2016-01-14 18:31:32,681 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@188e218d; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1b3cbc93; comparator=cascading.tuple.hadoop.util.TupleComparator@308064dd; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:32,681 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:32,681 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:32,681 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:32,694 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:32,695 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:32,695 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:32,695 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:32,695 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:32,696 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:32,696 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:32,696 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B34CC35AAD5145BFAC48D3E29E7EC246, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:32,699 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796292480_0001_1_00_000000_0
2016-01-14 18:31:32,699 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B34CC35AAD5145BFAC48D3E29E7EC246, mem on close (mb), free: 137, total: 372, max: 672
2016-01-14 18:31:32,699 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:32,699 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:32,699 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:32,699 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:32,701 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:32 Completed running task attempt: attempt_1452796292480_0001_1_00_000000_0
2016-01-14 18:31:32,704 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796292480_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:32,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796292480_0001_1_00, eventCount=1
2016-01-14 18:31:32,704 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:32,704 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B34CC35AAD5145BFAC48D3E29E7EC246, taskAttemptId=attempt_1452796292480_0001_1_00_000000_0, startTime=1452796292661, finishTime=1452796292704, timeTaken=43, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796292696, Process_Duration=3, Process_End_Time=1452796292699, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:32,705 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292480_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:32,705 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:32,705 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796292480_0001_1_00_000000_0
2016-01-14 18:31:32,706 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:32,706 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:TASK_FINISHED]: vertexName=B34CC35AAD5145BFAC48D3E29E7EC246, taskId=task_1452796292480_0001_1_00_000000, startTime=1452796292661, finishTime=1452796292705, timeTaken=44, status=SUCCEEDED, successfulAttemptID=attempt_1452796292480_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796292696, Process_Duration=3, Process_End_Time=1452796292699, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:32,706 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292480_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:32,706 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246] : 1
2016-01-14 18:31:32,706 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246]
2016-01-14 18:31:32,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:VERTEX_FINISHED]: vertexName=B34CC35AAD5145BFAC48D3E29E7EC246, vertexId=vertex_1452796292480_0001_1_00, initRequestedTime=1452796292638, initedTime=1452796292649, startRequestedTime=1452796292649, startedTime=1452796292649, finishTime=1452796292706, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796292696, Process_Duration=3, Process_End_Time=1452796292699, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796292661, firstTasksToStart=[ task_1452796292480_0001_1_00_000000 ], lastTaskFinishTime=1452796292704, lastTasksToFinish=[ task_1452796292480_0001_1_00_000000 ], minTaskDuration=43, maxTaskDuration=43, avgTaskDuration=43.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796292480_0001_1_00_000000 ], longestDurationTasks=[ task_1452796292480_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:32,707 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:32,707 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:32,707 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:32,707 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796292480_0001_00_000001, stoppedTime=1452796292707, exitStatus=0
2016-01-14 18:31:32,707 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:32,707 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] attempt: attempt_1452796292480_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:32,707 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A6C46F0CEBD844E1B0CC85C7FAB4F0AE
2016-01-14 18:31:32,707 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE]
2016-01-14 18:31:32,708 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:32,708 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796292480_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:32,708 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:TASK_STARTED]: vertexName=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, taskId=task_1452796292480_0001_1_01_000000, scheduledTime=1452796292708, launchTime=1452796292708
2016-01-14 18:31:32,708 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292480_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:32,708 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE]
2016-01-14 18:31:32,708 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:32,708 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: A6C46F0CEBD844E1B0CC85C7FAB4F0AE, VertexParallelism: 1, TaskAttemptID:attempt_1452796292480_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B34CC35AAD5145BFAC48D3E29E7EC246, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7CA0F97A6C874958B01EB1D50A299D0C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:32,708 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292480_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:32,708 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:32,708 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:32,709 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:32,709 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796292480_0001_00_000002
2016-01-14 18:31:32,709 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:32,709 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796292696, Process_Duration=3, Process_End_Time=1452796292699, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:32,709 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:32,709 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:32,709 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796292480_0001_00_000001
2016-01-14 18:31:32,710 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796292480_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:32,710 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796292480_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:32,710 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796292480_0001_00_000001, Exiting
2016-01-14 18:31:32,710 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796292480_0001_00_000001
2016-01-14 18:31:32,710 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796292480_0001_00_000001
2016-01-14 18:31:32,710 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796292480_0001_00_000002
2016-01-14 18:31:32,711 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796292480_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:32,711 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796292480_0001_00_000001 completed successfully
2016-01-14 18:31:32,711 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:32,711 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:32,714 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:32,714 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:32,714 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796292480_0001_00_000002, launchTime=1452796292714
2016-01-14 18:31:32,715 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796292480_0001_00_000002
2016-01-14 18:31:32,715 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796292480_0001_1_01_000000_0] to container: [container_1452796292480_0001_00_000002]
2016-01-14 18:31:32,715 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:32,716 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796292480_0001_00_000002 given task: attempt_1452796292480_0001_1_01_000000_0
2016-01-14 18:31:32,716 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796292480_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796292480_0001_1_01_000000_0
2016-01-14 18:31:32,716 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796292480_0001_1_01_000000_0] started. Is using containerId: [container_1452796292480_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:32,716 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, taskAttemptId=attempt_1452796292480_0001_1_01_000000_0, startTime=1452796292716, containerId=container_1452796292480_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796292480_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:32 Starting to run new task attempt: attempt_1452796292480_0001_1_01_000000_0
2016-01-14 18:31:32,716 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:32,716 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796292480_0001_1_01_000000_0
2016-01-14 18:31:32,717 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:32,716 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292480_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:32,717 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:32,717 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:32,717 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292480_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:32,717 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:32,717 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:32,717 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:32,717 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4] every/(1/1) ...latform/0_UNKNOWN_REPLACE, VertexName: A6C46F0CEBD844E1B0CC85C7FAB4F0AE, VertexParallelism: 1, TaskAttemptID:attempt_1452796292480_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B34CC35AAD5145BFAC48D3E29E7EC246, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7CA0F97A6C874958B01EB1D50A299D0C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:32,717 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:32,718 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796292480_0001_1_01_000000_0
2016-01-14 18:31:32,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:32,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,718 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796292480_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:32,718 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B34CC35AAD5145BFAC48D3E29E7EC246, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:32,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:32,719 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B34CC35AAD5145BFAC48D3E29E7EC246
2016-01-14 18:31:32,720 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,721 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:32,721 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:32,721 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B34CC35AAD5145BFAC48D3E29E7EC246
2016-01-14 18:31:32,721 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B34CC35AAD5145BFAC48D3E29E7EC246
2016-01-14 18:31:32,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7CA0F97A6C874958B01EB1D50A299D0C, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:32,722 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:32,723 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7CA0F97A6C874958B01EB1D50A299D0C
2016-01-14 18:31:32,723 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:32,723 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7CA0F97A6C874958B01EB1D50A299D0C
2016-01-14 18:31:32,723 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:32,729 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A6C46F0CEBD844E1B0CC85C7FAB4F0AE, ordinal: 1
2016-01-14 18:31:32,729 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6C46F0CEBD844E1B0CC85C7FAB4F0AE, mem on start (mb), free: 130, total: 372, max: 672
2016-01-14 18:31:32,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:32,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:32,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:32,730 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:32,730 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7CA0F97A6C874958B01EB1D50A299D0C
2016-01-14 18:31:32,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:32,731 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:32,731 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:32,731 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:32,731 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:32,731 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:32,732 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:32,732 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B34CC35AAD5145BFAC48D3E29E7EC246, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:32,732 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7CA0F97A6C874958B01EB1D50A299D0C, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:32,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:32,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B34CC35AAD5145BFAC48D3E29E7EC246 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:32,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B34CC35AAD5145BFAC48D3E29E7EC246
2016-01-14 18:31:32,732 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:32,733 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:32,733 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:32,733 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:32,733 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B34CC35AAD5145BFAC48D3E29E7EC246 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:32,734 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:32,734 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:32,734 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:32,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B34CC35AAD5145BFAC48D3E29E7EC246
2016-01-14 18:31:32,735 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:32,736 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796292480_0001_1_01_000000_0
2016-01-14 18:31:32,736 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796292480_0001_1_00_000000_0_10165, runDuration: 35174, ]
2016-01-14 18:31:32,738 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796292480_0001_1_00_000000_0_10165] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796292738, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:32,738 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B34CC35AAD5145BFAC48D3E29E7EC246
2016-01-14 18:31:32,738 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:32,738 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B34CC35AAD5145BFAC48D3E29E7EC246] #28 in 2ms
2016-01-14 18:31:32,738 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B34CC35AAD5145BFAC48D3E29E7EC246] #28, status:false, isInterrupted:false
2016-01-14 18:31:32,739 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:32,739 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:32,739 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:32,740 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:32,740 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:32,740 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 75983417F7F14E4D9D8D21B24F21ECDE
2016-01-14 18:31:32,740 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:32,740 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B34CC35AAD5145BFAC48D3E29E7EC246
2016-01-14 18:31:32,740 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:32,740 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE"], id: 7CA0F97A6C874958B01EB1D50A299D0C
2016-01-14 18:31:32,740 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_REPLACE"]
2016-01-14 18:31:32,741 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 75983417F7F14E4D9D8D21B24F21ECDE, for 1 inputs
2016-01-14 18:31:32,741 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A6C46F0CEBD844E1B0CC85C7FAB4F0AE, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:32,741 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6cbc0fb7; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2eabef26
2016-01-14 18:31:32,742 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:32,744 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:32,746 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:32,746 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:32,746 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796292480_0001_1_01_000000_0
2016-01-14 18:31:32,746 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796292480_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:32,747 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796292480_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:32,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796292741, Process_Duration=6, Process_End_Time=1452796292747, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:32,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:32,749 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, taskAttemptId=attempt_1452796292480_0001_1_01_000000_0, startTime=1452796292716, finishTime=1452796292749, timeTaken=33, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796292741, Process_Duration=6, Process_End_Time=1452796292747, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:32,750 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292480_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:32,750 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796292480_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:32,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:TASK_FINISHED]: vertexName=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, taskId=task_1452796292480_0001_1_01_000000, startTime=1452796292716, finishTime=1452796292750, timeTaken=34, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:32,751 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292480_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:32,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:32,751 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:32,751 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] : 1
2016-01-14 18:31:32,751 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] because task failed: task_1452796292480_0001_1_01_000000
2016-01-14 18:31:32,751 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:32,751 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:32,751 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE]
2016-01-14 18:31:32,751 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=7CA0F97A6C874958B01EB1D50A299D0C, vertexId=vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE]
2016-01-14 18:31:32,751 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A6C46F0CEBD844E1B0CC85C7FAB4F0AE, mem on close (mb), free: 125, total: 372, max: 672
2016-01-14 18:31:32,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, index=1
2016-01-14 18:31:32,752 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B34CC35AAD5145BFAC48D3E29E7EC246
2016-01-14 18:31:32,752 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:32,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:32,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:VERTEX_FINISHED]: vertexName=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, vertexId=vertex_1452796292480_0001_1_01, initRequestedTime=1452796292639, initedTime=1452796292647, startRequestedTime=1452796292652, startedTime=1452796292652, finishTime=1452796292752, timeTaken=100, status=FAILED, diagnostics=Task failed, taskId=task_1452796292480_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796292716, firstTasksToStart=[ task_1452796292480_0001_1_01_000000 ], lastTaskFinishTime=1452796292749, lastTasksToFinish=[ task_1452796292480_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:32,753 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:32,753 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796292480_0001_00_000002, stoppedTime=1452796292753, exitStatus=0
2016-01-14 18:31:32,753 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:32,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:32,753 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796292480_0001_1_01_000000_0 failed: true container: container_1452796292480_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:32,753 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:32,753 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:32,753 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796292480_0001_1
2016-01-14 18:31:32,753 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796292480_0001_1_00 [B34CC35AAD5145BFAC48D3E29E7EC246]
2016-01-14 18:31:32,754 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 7CA0F97A6C874958B01EB1D50A299D0C for vertex: vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE]
2016-01-14 18:31:32,753 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:32,754 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:32,754 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:32,754 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:32,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, sourceVertex=B34CC35AAD5145BFAC48D3E29E7EC246
2016-01-14 18:31:32,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, sourceVertex=7CA0F97A6C874958B01EB1D50A299D0C
2016-01-14 18:31:32,755 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796292480_0001_00_000002
2016-01-14 18:31:32,755 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796292480_0001_00_000002
2016-01-14 18:31:32,755 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796292480_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:32,755 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796292480_0001_00_000002 completed but with errors
2016-01-14 18:31:32,755 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796292480_0001_1, queueSize=0
2016-01-14 18:31:32,756 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:32,756 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292480_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796292480_0001_1, startTime=1452796292637, finishTime=1452796292754, timeTaken=117, status=FAILED, diagnostics=Vertex failed, vertexName=A6C46F0CEBD844E1B0CC85C7FAB4F0AE, vertexId=vertex_1452796292480_0001_1_01, diagnostics=[Task failed, taskId=task_1452796292480_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796292480_0001_1_01 [A6C46F0CEBD844E1B0CC85C7FAB4F0AE] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796292696, Process_Duration=3, Process_End_Time=1452796292699, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:32,756 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796292480_0001_1 finished with state: FAILED
2016-01-14 18:31:32,756 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796292480_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:32,756 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796292480_0001_00_000002
2016-01-14 18:31:32,756 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796292480_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:32,756 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292480_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:32 Completed Dag: dag_1452796292480_0001_1
2016-01-14 18:31:32,757 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796292480_0001_1, dagState=FAILED
2016-01-14 18:31:32,757 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796292480_0001_1_post
2016-01-14 18:31:32,757 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:32,757 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:32,757 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292480_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:32,769 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...latform/0_UNKNOWN_REPLACE, applicationId=application_1452796292480_0001
2016-01-14 18:31:32,770 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:32,770 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:32,770 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:32,770 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:32,771 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:32,771 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:32,771 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:32,771 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:32,772 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:32,772 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:32,772 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:32,772 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:32,772 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:32,777 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C28E337299B647BABEF8D5EE94C83390/B4D204F6E82343BAA7E68BC7C78F5FC4/.tez/application_1452796292480_0001
2016-01-14 18:31:32,777 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:32,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...latform/0_UNKNOWN_REPLACE
2016-01-14 18:31:32,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:32,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.318
2016-01-14 18:31:32,779 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_REPLACE
2016-01-14 18:31:32,809 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:32,809 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:32,851 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 18:31:32,858 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.020
2016-01-14 18:31:32,859 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:32,859 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:32,859 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:32,882 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:32,890 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP
2016-01-14 18:31:32,958 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:32,958 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:32,958 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP"]
2016-01-14 18:31:32,958 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:32,958 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:32,958 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:32,959 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-01-14 18:31:32,959 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:32,975 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:32,980 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:32,980 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:32,980 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:32,991 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513/.tez/application_1452796292980_0001 doesn't exist and is created
2016-01-14 18:31:33,007 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:33,008 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:33,008 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513/.tez/application_1452796292980_0001
2016-01-14 18:31:33,009 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796292980_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:33,020 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:33,020 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:33,021 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:33,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:33,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513/.tez/application_1452796292980_0001/localmode-log-dir
2016-01-14 18:31:33,022 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513/.tez/application_1452796292980_0001/localmode-log-dir/history.txt.appattempt_1452796292980_0001_000000, maxErrors=10
2016-01-14 18:31:33,022 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:33,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796292980_0001_000000, appSubmitTime=1452796293009, launchTime=1452796293009
2016-01-14 18:31:33,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:33,027 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:33,028 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:33,029 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:33,030 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:50987
2016-01-14 18:31:33,032 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:33,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796292980_0001_000000, startTime=1452796293033
2016-01-14 18:31:33,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:33,034 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:33,108 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:33,108 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:33,108 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1452796292980_0001, dagName=[7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-01-14 18:31:33,123 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-01-14 18:31:33,124 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796292980_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513/.tez/application_1452796292980_0001/localmode-log-dir/dag_1452796292980_0001_1.dot
2016-01-14 18:31:33,125 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513/.tez/application_1452796292980_0001/localmode-log-dir/dag_1452796292980_0001_1-tez-dag.pb.txt
2016-01-14 18:31:33,131 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796292980_0001_1
2016-01-14 18:31:33,131 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:33,131 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-01-14 18:31:33 Running Dag: dag_1452796292980_0001_1
2016-01-14 18:31:33,139 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796292980_0001_1, submitTime=1452796293123
2016-01-14 18:31:33,139 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:33,140 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CBCB0196E24E4D6390047E3DB4C41793
2016-01-14 18:31:33,140 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:33,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796292980_0001_1, initTime=1452796293140
2016-01-14 18:31:33,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:33,141 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796292980_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:33,141 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:DAG_STARTED]: dagID=dag_1452796292980_0001_1, startTime=1452796293141
2016-01-14 18:31:33,141 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:33,141 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:33,141 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796292980_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:33,142 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F28088A6C49743CF9631F677E557B39A : {37DEA521E85C4C88857033A8436EFAD7={InputName=37DEA521E85C4C88857033A8436EFAD7}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:33,142 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 37DEA521E85C4C88857033A8436EFAD7, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:33,142 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A]
2016-01-14 18:31:33,142 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A]
2016-01-14 18:31:33,142 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A]
2016-01-14 18:31:33,142 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A]. Starting root input initializers: 1
2016-01-14 18:31:33,142 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1452796292980_0001, dagName=[7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP
2016-01-14 18:31:33,143 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:33,143 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793]
2016-01-14 18:31:33,145 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:33,145 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793]
2016-01-14 18:31:33,145 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793]
2016-01-14 18:31:33,145 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793]
2016-01-14 18:31:33,145 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=139BA5062E5B4823A55F48171DA97387, vertexId=vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:33,145 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=139BA5062E5B4823A55F48171DA97387, vertexId=vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793]
2016-01-14 18:31:33,146 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 37DEA521E85C4C88857033A8436EFAD7 on vertex vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A]
2016-01-14 18:31:33,147 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=139BA5062E5B4823A55F48171DA97387, vertexName=CBCB0196E24E4D6390047E3DB4C41793, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:33,147 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=139BA5062E5B4823A55F48171DA97387, vertexId=vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793]
2016-01-14 18:31:33,148 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:33,148 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 37DEA521E85C4C88857033A8436EFAD7 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:33,148 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:33,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:33,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CBCB0196E24E4D6390047E3DB4C41793, vertexId=vertex_1452796292980_0001_1_01, initRequestedTime=1452796293143, initedTime=1452796293148, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:33,148 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:33,148 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:33,150 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:33,150 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 37DEA521E85C4C88857033A8436EFAD7 on vertex vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A]
2016-01-14 18:31:33,150 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:33,150 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {37DEA521E85C4C88857033A8436EFAD7=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:33,150 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A] parallelism set to 1
2016-01-14 18:31:33,150 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:33,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796292980_0001_1_00, eventCount=1
2016-01-14 18:31:33,151 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:33,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F28088A6C49743CF9631F677E557B39A, vertexId=vertex_1452796292980_0001_1_00, initRequestedTime=1452796293142, initedTime=1452796293151, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:33,151 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A] with distanceFromRoot: 0
2016-01-14 18:31:33,151 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:33,151 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F28088A6C49743CF9631F677E557B39A
2016-01-14 18:31:33,151 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A]
2016-01-14 18:31:33,152 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:33,152 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796292980_0001_1_00, startRequestedTime=1452796293151, startedTime=1452796293151
2016-01-14 18:31:33,152 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:33,152 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:33,152 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:TASK_STARTED]: vertexName=F28088A6C49743CF9631F677E557B39A, taskId=task_1452796292980_0001_1_00_000000, scheduledTime=1452796293152, launchTime=1452796293152
2016-01-14 18:31:33,153 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292980_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:33,153 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796292980_0001_1_00 for vertex: vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793] numStartedSources: 1 numSources: 1
2016-01-14 18:31:33,153 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793] with distanceFromRoot: 1
2016-01-14 18:31:33,153 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A]
2016-01-14 18:31:33,153 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F28088A6C49743CF9631F677E557B39A in vertex: CBCB0196E24E4D6390047E3DB4C41793
2016-01-14 18:31:33,153 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CBCB0196E24E4D6390047E3DB4C41793 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:33,153 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:33,154 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796292980_0001_1_01, startRequestedTime=1452796293153, startedTime=1452796293153
2016-01-14 18:31:33,154 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:33,154 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: F28088A6C49743CF9631F677E557B39A, VertexParallelism: 1, TaskAttemptID:attempt_1452796292980_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=37DEA521E85C4C88857033A8436EFAD7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CBCB0196E24E4D6390047E3DB4C41793, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:33,154 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292980_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:33,154 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:33,155 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:33,155 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:33,155 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:33,156 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796292980_0001_00_000001
2016-01-14 18:31:33,156 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:33,157 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796292980_0001_00_000001
2016-01-14 18:31:33,157 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:33,159 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:33,160 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:33,161 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796292980_0001_00_000001, launchTime=1452796293159
2016-01-14 18:31:33,161 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796292980_0001_00_000001
2016-01-14 18:31:33,162 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796292980_0001_1_00_000000_0] to container: [container_1452796292980_0001_00_000001]
2016-01-14 18:31:33,163 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:33,163 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796292980_0001_00_000001 given task: attempt_1452796292980_0001_1_00_000000_0
2016-01-14 18:31:33,163 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796292980_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796292980_0001_1_00_000000_0
2016-01-14 18:31:33,163 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796292980_0001_1_00_000000_0] started. Is using containerId: [container_1452796292980_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:33,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F28088A6C49743CF9631F677E557B39A, taskAttemptId=attempt_1452796292980_0001_1_00_000000_0, startTime=1452796293163, containerId=container_1452796292980_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796292980_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:33,165 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292980_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:33,165 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292980_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:33,164 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796292980_0001, with dag id: dag_1452796292980_0001_1
2016-01-14 18:31:33,165 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:33 Starting to run new task attempt: attempt_1452796292980_0001_1_00_000000_0
2016-01-14 18:31:33,167 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796292980_0001_1_00_000000_0
2016-01-14 18:31:33,167 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:33,167 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:33,167 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:33,167 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:33,167 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:33,167 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:33,168 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: F28088A6C49743CF9631F677E557B39A, VertexParallelism: 1, TaskAttemptID:attempt_1452796292980_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=37DEA521E85C4C88857033A8436EFAD7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CBCB0196E24E4D6390047E3DB4C41793, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:33,168 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:33,172 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796292980_0001_1_00_000000_0
2016-01-14 18:31:33,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:33,172 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,176 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796292980_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:33,179 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=37DEA521E85C4C88857033A8436EFAD7, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:33,180 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:33,181 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 37DEA521E85C4C88857033A8436EFAD7
2016-01-14 18:31:33,181 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 37DEA521E85C4C88857033A8436EFAD7
2016-01-14 18:31:33,183 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:33,183 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 37DEA521E85C4C88857033A8436EFAD7
2016-01-14 18:31:33,187 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 37DEA521E85C4C88857033A8436EFAD7
2016-01-14 18:31:33,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CBCB0196E24E4D6390047E3DB4C41793, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:33,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,191 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:33,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:33,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CBCB0196E24E4D6390047E3DB4C41793
2016-01-14 18:31:33,195 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:33,197 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CBCB0196E24E4D6390047E3DB4C41793
2016-01-14 18:31:33,198 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CBCB0196E24E4D6390047E3DB4C41793
2016-01-14 18:31:33,198 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F28088A6C49743CF9631F677E557B39A, ordinal: 0
2016-01-14 18:31:33,202 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F28088A6C49743CF9631F677E557B39A, mem on start (mb), free: 246, total: 371, max: 672
2016-01-14 18:31:33,202 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:33,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:33,203 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:33,203 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:33,203 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:33,203 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:33,203 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:33,203 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:33,204 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:33,204 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:33,204 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:33,204 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 37DEA521E85C4C88857033A8436EFAD7, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:33,204 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CBCB0196E24E4D6390047E3DB4C41793, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:33,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:33,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 37DEA521E85C4C88857033A8436EFAD7 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:33,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:33,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:33,204 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 37DEA521E85C4C88857033A8436EFAD7
2016-01-14 18:31:33,205 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:33,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 37DEA521E85C4C88857033A8436EFAD7
2016-01-14 18:31:33,205 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:33,205 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796292980_0001_1_00_000000_0
2016-01-14 18:31:33,214 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:33,214 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 37DEA521E85C4C88857033A8436EFAD7
2016-01-14 18:31:33,214 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 05B30BAAABFC49F59CBB4D4CEF583A4D
2016-01-14 18:31:33,215 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 05B30BAAABFC49F59CBB4D4CEF583A4D
2016-01-14 18:31:33,215 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@28e2b4c2; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@3bc94186; comparator=cascading.tuple.hadoop.util.TupleComparator@19ece314; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:33,216 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:33,216 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:33,216 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:33,215 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:33,222 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@29954e32
2016-01-14 18:31:33,223 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@29954e32
2016-01-14 18:31:33,223 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:33,223 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:33,233 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:33,233 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:33,233 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:33,233 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:33,233 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:33,233 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:33,233 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:33,234 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F28088A6C49743CF9631F677E557B39A, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:33,237 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796292980_0001_1_00_000000_0
2016-01-14 18:31:33,237 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F28088A6C49743CF9631F677E557B39A, mem on close (mb), free: 144, total: 371, max: 672
2016-01-14 18:31:33,237 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:33,237 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:33,237 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:33,237 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:33,239 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:33 Completed running task attempt: attempt_1452796292980_0001_1_00_000000_0
2016-01-14 18:31:33,243 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796292980_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:33,243 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796292980_0001_1_00, eventCount=1
2016-01-14 18:31:33,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F28088A6C49743CF9631F677E557B39A, taskAttemptId=attempt_1452796292980_0001_1_00_000000_0, startTime=1452796293163, finishTime=1452796293243, timeTaken=80, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796293234, Process_Duration=3, Process_End_Time=1452796293237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:33,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:33,244 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292980_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:33,244 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796292980_0001_1_00_000000_0
2016-01-14 18:31:33,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:33,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:TASK_FINISHED]: vertexName=F28088A6C49743CF9631F677E557B39A, taskId=task_1452796292980_0001_1_00_000000, startTime=1452796293163, finishTime=1452796293244, timeTaken=81, status=SUCCEEDED, successfulAttemptID=attempt_1452796292980_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796293234, Process_Duration=3, Process_End_Time=1452796293237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:33,245 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292980_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:33,245 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A] : 1
2016-01-14 18:31:33,245 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A]
2016-01-14 18:31:33,245 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:VERTEX_FINISHED]: vertexName=F28088A6C49743CF9631F677E557B39A, vertexId=vertex_1452796292980_0001_1_00, initRequestedTime=1452796293142, initedTime=1452796293151, startRequestedTime=1452796293151, startedTime=1452796293151, finishTime=1452796293245, timeTaken=94, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796293234, Process_Duration=3, Process_End_Time=1452796293237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796293163, firstTasksToStart=[ task_1452796292980_0001_1_00_000000 ], lastTaskFinishTime=1452796293243, lastTasksToFinish=[ task_1452796292980_0001_1_00_000000 ], minTaskDuration=80, maxTaskDuration=80, avgTaskDuration=80.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796292980_0001_1_00_000000 ], longestDurationTasks=[ task_1452796292980_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:33,246 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:33,246 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793] attempt: attempt_1452796292980_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:33,246 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CBCB0196E24E4D6390047E3DB4C41793
2016-01-14 18:31:33,246 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793]
2016-01-14 18:31:33,246 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:33,246 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:33,246 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:TASK_STARTED]: vertexName=CBCB0196E24E4D6390047E3DB4C41793, taskId=task_1452796292980_0001_1_01_000000, scheduledTime=1452796293246, launchTime=1452796293246
2016-01-14 18:31:33,247 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292980_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:33,247 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793]
2016-01-14 18:31:33,247 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:33,247 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:33,247 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: CBCB0196E24E4D6390047E3DB4C41793, VertexParallelism: 1, TaskAttemptID:attempt_1452796292980_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F28088A6C49743CF9631F677E557B39A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=139BA5062E5B4823A55F48171DA97387, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:33,247 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292980_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:33,248 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:33,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796292980_0001_00_000001, stoppedTime=1452796293248, exitStatus=0
2016-01-14 18:31:33,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:33,248 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:33,248 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:33,248 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796292980_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:33,248 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:33,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796293234, Process_Duration=3, Process_End_Time=1452796293237, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:33,249 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796292980_0001_00_000002
2016-01-14 18:31:33,249 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:33,249 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:33,249 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:33,249 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796292980_0001_00_000001
2016-01-14 18:31:33,250 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796292980_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:33,250 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796292980_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:33,250 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796292980_0001_00_000001, Exiting
2016-01-14 18:31:33,250 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796292980_0001_00_000001
2016-01-14 18:31:33,250 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796292980_0001_00_000001
2016-01-14 18:31:33,251 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796292980_0001_00_000002
2016-01-14 18:31:33,251 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796292980_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:33,251 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796292980_0001_00_000001 completed successfully
2016-01-14 18:31:33,251 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:33,252 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:33,254 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:33,254 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796292980_0001_00_000002, launchTime=1452796293254
2016-01-14 18:31:33,254 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:33,255 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796292980_0001_00_000002
2016-01-14 18:31:33,255 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796292980_0001_1_01_000000_0] to container: [container_1452796292980_0001_00_000002]
2016-01-14 18:31:33,255 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:33,255 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796292980_0001_00_000002 given task: attempt_1452796292980_0001_1_01_000000_0
2016-01-14 18:31:33,255 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796292980_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796292980_0001_1_01_000000_0
2016-01-14 18:31:33,256 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796292980_0001_1_01_000000_0] started. Is using containerId: [container_1452796292980_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:33,256 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CBCB0196E24E4D6390047E3DB4C41793, taskAttemptId=attempt_1452796292980_0001_1_01_000000_0, startTime=1452796293255, containerId=container_1452796292980_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796292980_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:33 Starting to run new task attempt: attempt_1452796292980_0001_1_01_000000_0
2016-01-14 18:31:33,256 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292980_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:33,256 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:33,256 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796292980_0001_1_01_000000_0
2016-01-14 18:31:33,256 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:33,256 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292980_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:33,256 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:33,256 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:33,256 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:33,257 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:33,257 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:33,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513] every/(1/1) ...lyplatform/0_UNKNOWN_SWAP, VertexName: CBCB0196E24E4D6390047E3DB4C41793, VertexParallelism: 1, TaskAttemptID:attempt_1452796292980_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F28088A6C49743CF9631F677E557B39A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=139BA5062E5B4823A55F48171DA97387, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:33,257 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:33,257 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796292980_0001_1_01_000000_0
2016-01-14 18:31:33,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:33,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,258 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F28088A6C49743CF9631F677E557B39A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:33,259 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796292980_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:33,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:33,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F28088A6C49743CF9631F677E557B39A
2016-01-14 18:31:33,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,260 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=139BA5062E5B4823A55F48171DA97387, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:33,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:33,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 139BA5062E5B4823A55F48171DA97387
2016-01-14 18:31:33,261 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:33,261 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 139BA5062E5B4823A55F48171DA97387
2016-01-14 18:31:33,261 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:33,261 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:33,261 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F28088A6C49743CF9631F677E557B39A
2016-01-14 18:31:33,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F28088A6C49743CF9631F677E557B39A
2016-01-14 18:31:33,263 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:33,268 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CBCB0196E24E4D6390047E3DB4C41793, ordinal: 1
2016-01-14 18:31:33,268 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CBCB0196E24E4D6390047E3DB4C41793, mem on start (mb), free: 137, total: 371, max: 672
2016-01-14 18:31:33,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:33,268 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:33,269 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:33,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 139BA5062E5B4823A55F48171DA97387
2016-01-14 18:31:33,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:33,270 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:33,270 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:33,270 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:33,270 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:33,270 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:33,271 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:33,271 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 139BA5062E5B4823A55F48171DA97387, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:33,271 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F28088A6C49743CF9631F677E557B39A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:33,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:33,271 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F28088A6C49743CF9631F677E557B39A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:33,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F28088A6C49743CF9631F677E557B39A
2016-01-14 18:31:33,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:33,272 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:33,272 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:33,273 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:33,273 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F28088A6C49743CF9631F677E557B39A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:33,273 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:33,273 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:33,273 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:33,274 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F28088A6C49743CF9631F677E557B39A
2016-01-14 18:31:33,276 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:33,276 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796292980_0001_1_01_000000_0
2016-01-14 18:31:33,279 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796292980_0001_1_00_000000_0_10171, runDuration: 47067, ]
2016-01-14 18:31:33,280 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796292980_0001_1_00_000000_0_10171] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796293280, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:33,281 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F28088A6C49743CF9631F677E557B39A
2016-01-14 18:31:33,281 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:33,281 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F28088A6C49743CF9631F677E557B39A] #29 in 2ms
2016-01-14 18:31:33,281 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:33,281 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F28088A6C49743CF9631F677E557B39A] #29, status:false, isInterrupted:false
2016-01-14 18:31:33,282 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:33,282 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:33,282 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:33,282 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:33,282 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 05B30BAAABFC49F59CBB4D4CEF583A4D
2016-01-14 18:31:33,283 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:33,283 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F28088A6C49743CF9631F677E557B39A
2016-01-14 18:31:33,283 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:33,283 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP"], id: 139BA5062E5B4823A55F48171DA97387
2016-01-14 18:31:33,284 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_UNKNOWN_SWAP"]
2016-01-14 18:31:33,284 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 05B30BAAABFC49F59CBB4D4CEF583A4D, for 1 inputs
2016-01-14 18:31:33,284 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CBCB0196E24E4D6390047E3DB4C41793, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:33,285 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@19eebb06; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@8389437
2016-01-14 18:31:33,288 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:33,288 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:33,289 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796292980_0001_1_01_000000_0
2016-01-14 18:31:33,289 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796292980_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:33,290 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796292980_0001_1_01_000000_0
2016-01-14 18:31:33,290 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CBCB0196E24E4D6390047E3DB4C41793, mem on close (mb), free: 132, total: 371, max: 672
2016-01-14 18:31:33,290 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F28088A6C49743CF9631F677E557B39A
2016-01-14 18:31:33,290 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:33,291 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:33,291 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:33,291 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:33,291 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:33 Completed running task attempt: attempt_1452796292980_0001_1_01_000000_0
2016-01-14 18:31:33,292 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796292980_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:33,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:33,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CBCB0196E24E4D6390047E3DB4C41793, taskAttemptId=attempt_1452796292980_0001_1_01_000000_0, startTime=1452796293255, finishTime=1452796293293, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796293285, Process_Duration=4, Process_End_Time=1452796293289, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:33,293 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796292980_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:33,293 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796292980_0001_1_01_000000_0
2016-01-14 18:31:33,293 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:33,294 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:33,294 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:TASK_FINISHED]: vertexName=CBCB0196E24E4D6390047E3DB4C41793, taskId=task_1452796292980_0001_1_01_000000, startTime=1452796293255, finishTime=1452796293293, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1452796292980_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796293285, Process_Duration=4, Process_End_Time=1452796293289, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:33,294 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796292980_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:33,294 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793] : 1
2016-01-14 18:31:33,294 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793]
2016-01-14 18:31:33,295 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:33,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:VERTEX_FINISHED]: vertexName=CBCB0196E24E4D6390047E3DB4C41793, vertexId=vertex_1452796292980_0001_1_01, initRequestedTime=1452796293143, initedTime=1452796293148, startRequestedTime=1452796293153, startedTime=1452796293153, finishTime=1452796293294, timeTaken=141, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796293285, Process_Duration=4, Process_End_Time=1452796293289, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796293255, firstTasksToStart=[ task_1452796292980_0001_1_01_000000 ], lastTaskFinishTime=1452796293293, lastTasksToFinish=[ task_1452796292980_0001_1_01_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796292980_0001_1_01_000000 ], longestDurationTasks=[ task_1452796292980_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:33,295 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:33,296 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:33,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796292980_0001_00_000002, stoppedTime=1452796293296, exitStatus=0
2016-01-14 18:31:33,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:33,296 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:33,296 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:33,296 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796292980_0001_1
2016-01-14 18:31:33,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796292980_0001_1
2016-01-14 18:31:33,297 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796292980_0001_1_00 [F28088A6C49743CF9631F677E557B39A]
2016-01-14 18:31:33,297 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 139BA5062E5B4823A55F48171DA97387 for vertex: vertex_1452796292980_0001_1_01 [CBCB0196E24E4D6390047E3DB4C41793]
2016-01-14 18:31:33,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389021696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796293285, Process_Duration=4, Process_End_Time=1452796293289, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:33,298 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:33,298 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:33,299 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796292980_0001_00_000002
2016-01-14 18:31:33,299 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796292980_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:33,300 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796292980_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:33,300 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796292980_0001_00_000002, Exiting
2016-01-14 18:31:33,300 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796292980_0001_00_000002
2016-01-14 18:31:33,300 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796292980_0001_00_000002
2016-01-14 18:31:33,300 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796292980_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:33,300 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796292980_0001_00_000002 completed successfully
2016-01-14 18:31:33,307 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796292980_0001_1, queueSize=0
2016-01-14 18:31:33,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:33,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796292980_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796292980_0001_1, startTime=1452796293141, finishTime=1452796293305, timeTaken=164, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=516, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=778043392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592586519, Process_Duration=7, Process_End_Time=2905592586526, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:33,307 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796292980_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:33,307 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796292980_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:33,307 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796292980_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:33,307 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796292980_0001_00_000002
2016-01-14 18:31:33 Completed Dag: dag_1452796292980_0001_1
2016-01-14 18:31:33,308 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796292980_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:33,308 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796292980_0001_1_post
2016-01-14 18:31:33,308 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:33,308 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:33,314 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_UNKNOWN_SWAP, applicationId=application_1452796292980_0001
2016-01-14 18:31:33,315 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:33,315 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:33,315 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:33,315 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:33,315 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:33,316 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:33,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:33,316 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:33,316 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:33,317 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:33,317 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:33,317 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:33,317 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:33,321 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/7FAADDD661F040679F0F692917CA14B6/F96FF3ACE581490799F375DC95A77513/.tez/application_1452796292980_0001
2016-01-14 18:31:33,322 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.363
2016-01-14 18:31:33,387 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_UNKNOWN_SWAP
2016-01-14 18:31:33,417 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:33,417 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:33,452 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 18:31:33,459 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 18:31:33,459 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:33,460 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:33,460 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:33,484 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:33,492 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1
2016-01-14 18:31:33,537 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:33,537 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:33,537 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1"]
2016-01-14 18:31:33,537 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:33,537 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:33,537 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:33,538 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 18:31:33,538 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:33,550 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:33,554 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:33,555 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:33,555 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:33,564 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA/.tez/application_1452796293555_0001 doesn't exist and is created
2016-01-14 18:31:33,578 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:33,578 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:33,579 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA/.tez/application_1452796293555_0001
2016-01-14 18:31:33,580 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796293555_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:33,590 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:33,591 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:33,592 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:33,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:33,592 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA/.tez/application_1452796293555_0001/localmode-log-dir
2016-01-14 18:31:33,592 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA/.tez/application_1452796293555_0001/localmode-log-dir/history.txt.appattempt_1452796293555_0001_000000, maxErrors=10
2016-01-14 18:31:33,592 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:33,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796293555_0001_000000, appSubmitTime=1452796293580, launchTime=1452796293580
2016-01-14 18:31:33,595 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:33,596 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:33,596 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:33,596 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:33,603 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:40976
2016-01-14 18:31:33,603 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:33,605 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796293555_0001_000000, startTime=1452796293605
2016-01-14 18:31:33,605 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:33,605 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:33,679 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:33,679 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:33,679 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1452796293555_0001, dagName=[C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 18:31:33,695 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 18:31:33,696 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796293555_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA/.tez/application_1452796293555_0001/localmode-log-dir/dag_1452796293555_0001_1.dot
2016-01-14 18:31:33,696 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA/.tez/application_1452796293555_0001/localmode-log-dir/dag_1452796293555_0001_1-tez-dag.pb.txt
2016-01-14 18:31:33,702 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796293555_0001_1
2016-01-14 18:31:33,702 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:33,703 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 18:31:33 Running Dag: dag_1452796293555_0001_1
2016-01-14 18:31:33,711 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:33,711 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796293555_0001_1, submitTime=1452796293695
2016-01-14 18:31:33,712 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C5C25D56F95D4CD2A19A23083E22D9E7
2016-01-14 18:31:33,712 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:33,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:33,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796293555_0001_1, initTime=1452796293712
2016-01-14 18:31:33,713 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796293555_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:33,713 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:DAG_STARTED]: dagID=dag_1452796293555_0001_1, startTime=1452796293713
2016-01-14 18:31:33,713 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:33,713 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:33,714 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796293555_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:33,714 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D5E03B5D77A84DCA8F0AB28AFB1369F3 : {713AAB1C423B4399BAB96E7499C729FA={InputName=713AAB1C423B4399BAB96E7499C729FA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:33,714 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 713AAB1C423B4399BAB96E7499C729FA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:33,714 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3]
2016-01-14 18:31:33,714 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1452796293555_0001, dagName=[C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA] every/(1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 18:31:33,714 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3]
2016-01-14 18:31:33,714 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3]
2016-01-14 18:31:33,714 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3]. Starting root input initializers: 1
2016-01-14 18:31:33,715 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:33,715 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7]
2016-01-14 18:31:33,718 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:33,718 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7]
2016-01-14 18:31:33,718 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7]
2016-01-14 18:31:33,718 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7]
2016-01-14 18:31:33,719 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B5D5B4AC1E5F4C338DE3BAB830B4BAEF, vertexId=vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:33,719 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B5D5B4AC1E5F4C338DE3BAB830B4BAEF, vertexId=vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7]
2016-01-14 18:31:33,721 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B5D5B4AC1E5F4C338DE3BAB830B4BAEF, vertexName=C5C25D56F95D4CD2A19A23083E22D9E7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:33,721 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B5D5B4AC1E5F4C338DE3BAB830B4BAEF, vertexId=vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7]
2016-01-14 18:31:33,721 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C5C25D56F95D4CD2A19A23083E22D9E7, vertexId=vertex_1452796293555_0001_1_01, initRequestedTime=1452796293715, initedTime=1452796293721, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:33,721 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:33,722 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:33,722 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 713AAB1C423B4399BAB96E7499C729FA on vertex vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3]
2016-01-14 18:31:33,724 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:33,724 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 713AAB1C423B4399BAB96E7499C729FA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:33,725 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:33,725 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:33,726 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:33,727 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 713AAB1C423B4399BAB96E7499C729FA on vertex vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3]
2016-01-14 18:31:33,727 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:33,727 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {713AAB1C423B4399BAB96E7499C729FA=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:33,727 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3] parallelism set to 1
2016-01-14 18:31:33,727 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:33,728 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796293555_0001_1_00, eventCount=1
2016-01-14 18:31:33,728 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:33,728 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D5E03B5D77A84DCA8F0AB28AFB1369F3, vertexId=vertex_1452796293555_0001_1_00, initRequestedTime=1452796293714, initedTime=1452796293728, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:33,729 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3] with distanceFromRoot: 0
2016-01-14 18:31:33,729 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:33,729 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D5E03B5D77A84DCA8F0AB28AFB1369F3
2016-01-14 18:31:33,729 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3]
2016-01-14 18:31:33,730 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:33,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796293555_0001_1_00, startRequestedTime=1452796293729, startedTime=1452796293729
2016-01-14 18:31:33,730 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:33,730 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:33,731 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:TASK_STARTED]: vertexName=D5E03B5D77A84DCA8F0AB28AFB1369F3, taskId=task_1452796293555_0001_1_00_000000, scheduledTime=1452796293730, launchTime=1452796293730
2016-01-14 18:31:33,731 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796293555_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:33,731 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796293555_0001_1_00 for vertex: vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] numStartedSources: 1 numSources: 1
2016-01-14 18:31:33,731 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] with distanceFromRoot: 1
2016-01-14 18:31:33,731 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3]
2016-01-14 18:31:33,732 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D5E03B5D77A84DCA8F0AB28AFB1369F3 in vertex: C5C25D56F95D4CD2A19A23083E22D9E7
2016-01-14 18:31:33,732 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C5C25D56F95D4CD2A19A23083E22D9E7 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:33,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:33,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796293555_0001_1_01, startRequestedTime=1452796293732, startedTime=1452796293732
2016-01-14 18:31:33,733 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:33,733 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: D5E03B5D77A84DCA8F0AB28AFB1369F3, VertexParallelism: 1, TaskAttemptID:attempt_1452796293555_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=713AAB1C423B4399BAB96E7499C729FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C5C25D56F95D4CD2A19A23083E22D9E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:33,733 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796293555_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:33,733 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:33,734 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:33,734 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:33,734 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796293555_0001, with dag id: dag_1452796293555_0001_1
2016-01-14 18:31:33,734 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:33,735 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796293555_0001_00_000001
2016-01-14 18:31:33,735 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:33,737 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796293555_0001_00_000001
2016-01-14 18:31:33,737 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:33,737 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:33,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796293555_0001_00_000001, launchTime=1452796293737
2016-01-14 18:31:33,738 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:33,738 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796293555_0001_00_000001
2016-01-14 18:31:33,738 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796293555_0001_1_00_000000_0] to container: [container_1452796293555_0001_00_000001]
2016-01-14 18:31:33,739 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:33,739 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796293555_0001_00_000001 given task: attempt_1452796293555_0001_1_00_000000_0
2016-01-14 18:31:33,739 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796293555_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796293555_0001_1_00_000000_0
2016-01-14 18:31:33,739 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796293555_0001_1_00_000000_0] started. Is using containerId: [container_1452796293555_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:33 Starting to run new task attempt: attempt_1452796293555_0001_1_00_000000_0
2016-01-14 18:31:33,740 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796293555_0001_1_00_000000_0
2016-01-14 18:31:33,740 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:33,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:33,740 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:33,740 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:33,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D5E03B5D77A84DCA8F0AB28AFB1369F3, taskAttemptId=attempt_1452796293555_0001_1_00_000000_0, startTime=1452796293739, containerId=container_1452796293555_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796293555_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:33,740 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796293555_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:33,740 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796293555_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:33,740 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:33,740 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:33,740 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:33,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: D5E03B5D77A84DCA8F0AB28AFB1369F3, VertexParallelism: 1, TaskAttemptID:attempt_1452796293555_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=713AAB1C423B4399BAB96E7499C729FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C5C25D56F95D4CD2A19A23083E22D9E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:33,741 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:33,741 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796293555_0001_1_00_000000_0
2016-01-14 18:31:33,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:33,741 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,742 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796293555_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:33,742 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C5C25D56F95D4CD2A19A23083E22D9E7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:33,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:33,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C5C25D56F95D4CD2A19A23083E22D9E7
2016-01-14 18:31:33,743 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=713AAB1C423B4399BAB96E7499C729FA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:33,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:33,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 713AAB1C423B4399BAB96E7499C729FA
2016-01-14 18:31:33,744 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 713AAB1C423B4399BAB96E7499C729FA
2016-01-14 18:31:33,744 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:33,745 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:33,746 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C5C25D56F95D4CD2A19A23083E22D9E7
2016-01-14 18:31:33,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C5C25D56F95D4CD2A19A23083E22D9E7
2016-01-14 18:31:33,747 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:33,747 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 713AAB1C423B4399BAB96E7499C729FA
2016-01-14 18:31:33,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 713AAB1C423B4399BAB96E7499C729FA
2016-01-14 18:31:33,749 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D5E03B5D77A84DCA8F0AB28AFB1369F3, ordinal: 0
2016-01-14 18:31:33,749 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5E03B5D77A84DCA8F0AB28AFB1369F3, mem on start (mb), free: 234, total: 371, max: 672
2016-01-14 18:31:33,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:33,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:33,749 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:33,749 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:33,750 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:33,750 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:33,750 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:33,750 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:33,750 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:33,751 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 713AAB1C423B4399BAB96E7499C729FA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:33,751 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C5C25D56F95D4CD2A19A23083E22D9E7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:33,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:33,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 713AAB1C423B4399BAB96E7499C729FA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:33,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:33,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:33,751 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 713AAB1C423B4399BAB96E7499C729FA
2016-01-14 18:31:33,752 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:33,752 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 713AAB1C423B4399BAB96E7499C729FA
2016-01-14 18:31:33,753 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:33,753 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796293555_0001_1_00_000000_0
2016-01-14 18:31:33,753 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:33,754 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@69119ead
2016-01-14 18:31:33,755 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@69119ead
2016-01-14 18:31:33,755 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:33,755 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:33,757 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:33,757 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 713AAB1C423B4399BAB96E7499C729FA
2016-01-14 18:31:33,757 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 579819F5447A4008A0635900C96279B9
2016-01-14 18:31:33,758 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 579819F5447A4008A0635900C96279B9
2016-01-14 18:31:33,758 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@34245cfc; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@3f54bf22; comparator=cascading.tuple.hadoop.util.TupleComparator@54d071a9; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:33,759 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:33,759 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:33,759 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:33,772 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:33,773 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:33,773 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:33,773 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:33,773 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:33,774 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:33,774 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:33,775 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D5E03B5D77A84DCA8F0AB28AFB1369F3, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:33,777 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796293555_0001_1_00_000000_0
2016-01-14 18:31:33,777 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D5E03B5D77A84DCA8F0AB28AFB1369F3, mem on close (mb), free: 132, total: 371, max: 672
2016-01-14 18:31:33,778 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:33,778 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:33,778 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:33,778 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:33,780 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:33 Completed running task attempt: attempt_1452796293555_0001_1_00_000000_0
2016-01-14 18:31:33,783 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796293555_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:33,784 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796293775, Process_Duration=2, Process_End_Time=1452796293777, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:33,784 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796293555_0001_1_00, eventCount=1
2016-01-14 18:31:33,784 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:33,784 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:33,784 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796293555_0001_00_000001
2016-01-14 18:31:33,784 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:33,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D5E03B5D77A84DCA8F0AB28AFB1369F3, taskAttemptId=attempt_1452796293555_0001_1_00_000000_0, startTime=1452796293739, finishTime=1452796293784, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796293775, Process_Duration=2, Process_End_Time=1452796293777, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:33,785 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796293555_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:33,785 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796293555_0001_1_00_000000_0
2016-01-14 18:31:33,785 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:33,785 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:TASK_FINISHED]: vertexName=D5E03B5D77A84DCA8F0AB28AFB1369F3, taskId=task_1452796293555_0001_1_00_000000, startTime=1452796293739, finishTime=1452796293785, timeTaken=46, status=SUCCEEDED, successfulAttemptID=attempt_1452796293555_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796293775, Process_Duration=2, Process_End_Time=1452796293777, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:33,785 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796293555_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:33,786 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3] : 1
2016-01-14 18:31:33,786 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3]
2016-01-14 18:31:33,786 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:33,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:VERTEX_FINISHED]: vertexName=D5E03B5D77A84DCA8F0AB28AFB1369F3, vertexId=vertex_1452796293555_0001_1_00, initRequestedTime=1452796293714, initedTime=1452796293728, startRequestedTime=1452796293729, startedTime=1452796293729, finishTime=1452796293786, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796293775, Process_Duration=2, Process_End_Time=1452796293777, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796293739, firstTasksToStart=[ task_1452796293555_0001_1_00_000000 ], lastTaskFinishTime=1452796293784, lastTasksToFinish=[ task_1452796293555_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796293555_0001_1_00_000000 ], longestDurationTasks=[ task_1452796293555_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:33,786 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:33,786 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] attempt: attempt_1452796293555_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:33,787 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C5C25D56F95D4CD2A19A23083E22D9E7
2016-01-14 18:31:33,787 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7]
2016-01-14 18:31:33,787 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:33,787 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:33,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:TASK_STARTED]: vertexName=C5C25D56F95D4CD2A19A23083E22D9E7, taskId=task_1452796293555_0001_1_01_000000, scheduledTime=1452796293787, launchTime=1452796293787
2016-01-14 18:31:33,787 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796293555_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:33,788 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:33,788 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7]
2016-01-14 18:31:33,788 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:33,788 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: C5C25D56F95D4CD2A19A23083E22D9E7, VertexParallelism: 1, TaskAttemptID:attempt_1452796293555_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5E03B5D77A84DCA8F0AB28AFB1369F3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B5D5B4AC1E5F4C338DE3BAB830B4BAEF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:33,788 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796293555_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:33,789 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:33,789 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796293555_0001_00_000001, stoppedTime=1452796293789, exitStatus=0
2016-01-14 18:31:33,789 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:33,789 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:33,790 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796293555_0001_00_000002
2016-01-14 18:31:33,790 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:33,790 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796293555_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:33,790 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:33,792 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796293555_0001_00_000002
2016-01-14 18:31:33,792 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:33,793 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:33,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:33,794 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796293555_0001_00_000002, launchTime=1452796293793
2016-01-14 18:31:33,794 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796293555_0001_00_000002
2016-01-14 18:31:33,795 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796293555_0001_1_01_000000_0] to container: [container_1452796293555_0001_00_000002]
2016-01-14 18:31:33,795 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:33,795 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796293555_0001_00_000002 given task: attempt_1452796293555_0001_1_01_000000_0
2016-01-14 18:31:33,795 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796293555_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:33,795 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796293555_0001_1_01_000000_0] started. Is using containerId: [container_1452796293555_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:33,795 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796293555_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796293555_0001_1_01_000000_0
2016-01-14 18:31:33,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C5C25D56F95D4CD2A19A23083E22D9E7, taskAttemptId=attempt_1452796293555_0001_1_01_000000_0, startTime=1452796293795, containerId=container_1452796293555_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796293555_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:33,795 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796293555_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:33,796 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796293555_0001_00_000001, Exiting
2016-01-14 18:31:33,796 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796293555_0001_00_000001
2016-01-14 18:31:33,796 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796293555_0001_00_000001
2016-01-14 18:31:33,796 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796293555_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:33,796 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796293555_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:33,795 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:33 Starting to run new task attempt: attempt_1452796293555_0001_1_01_000000_0
2016-01-14 18:31:33,797 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796293555_0001_1_01_000000_0
2016-01-14 18:31:33,797 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:33,797 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:33,797 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:33,797 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:33,797 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:33,798 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:33,798 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:33,798 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796293555_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:33,798 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796293555_0001_00_000001 completed successfully
2016-01-14 18:31:33,798 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:33,798 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:33,798 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA] every/(1/1) ...emblyplatform/0_VALUES_-1, VertexName: C5C25D56F95D4CD2A19A23083E22D9E7, VertexParallelism: 1, TaskAttemptID:attempt_1452796293555_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D5E03B5D77A84DCA8F0AB28AFB1369F3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B5D5B4AC1E5F4C338DE3BAB830B4BAEF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:33,799 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:33,799 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796293555_0001_1_01_000000_0
2016-01-14 18:31:33,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:33,799 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,800 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796293555_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:33,800 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D5E03B5D77A84DCA8F0AB28AFB1369F3, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:33,800 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B5D5B4AC1E5F4C338DE3BAB830B4BAEF, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:33,800 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:33,801 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D5E03B5D77A84DCA8F0AB28AFB1369F3
2016-01-14 18:31:33,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:33,802 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B5D5B4AC1E5F4C338DE3BAB830B4BAEF
2016-01-14 18:31:33,802 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:33,803 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B5D5B4AC1E5F4C338DE3BAB830B4BAEF
2016-01-14 18:31:33,803 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:33,803 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:33,803 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:33,804 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D5E03B5D77A84DCA8F0AB28AFB1369F3
2016-01-14 18:31:33,804 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D5E03B5D77A84DCA8F0AB28AFB1369F3
2016-01-14 18:31:33,811 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:33,812 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B5D5B4AC1E5F4C338DE3BAB830B4BAEF
2016-01-14 18:31:33,813 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C5C25D56F95D4CD2A19A23083E22D9E7, ordinal: 1
2016-01-14 18:31:33,813 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C5C25D56F95D4CD2A19A23083E22D9E7, mem on start (mb), free: 124, total: 371, max: 672
2016-01-14 18:31:33,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:33,813 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:33,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:33,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:33,814 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:33,814 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:33,814 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:33,814 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:33,815 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:33,815 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:33,815 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B5D5B4AC1E5F4C338DE3BAB830B4BAEF, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:33,815 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D5E03B5D77A84DCA8F0AB28AFB1369F3, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:33,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:33,815 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D5E03B5D77A84DCA8F0AB28AFB1369F3
2016-01-14 18:31:33,816 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:33,816 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:33,816 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D5E03B5D77A84DCA8F0AB28AFB1369F3 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:33,817 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:33,817 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:33,817 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:33,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D5E03B5D77A84DCA8F0AB28AFB1369F3
2016-01-14 18:31:33,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D5E03B5D77A84DCA8F0AB28AFB1369F3 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:33,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:33,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:33,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:33,818 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796293555_0001_1_01_000000_0
2016-01-14 18:31:33,823 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796293555_0001_1_00_000000_0_10176, runDuration: 38399, ]
2016-01-14 18:31:33,824 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796293555_0001_1_00_000000_0_10176] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796293824, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:33,824 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D5E03B5D77A84DCA8F0AB28AFB1369F3
2016-01-14 18:31:33,824 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:33,825 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D5E03B5D77A84DCA8F0AB28AFB1369F3] #30 in 2ms
2016-01-14 18:31:33,825 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D5E03B5D77A84DCA8F0AB28AFB1369F3] #30, status:false, isInterrupted:false
2016-01-14 18:31:33,825 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:33,825 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:33,826 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:33,826 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:33,826 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 579819F5447A4008A0635900C96279B9
2016-01-14 18:31:33,826 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:33,826 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:33,826 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D5E03B5D77A84DCA8F0AB28AFB1369F3
2016-01-14 18:31:33,826 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:33,826 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1"], id: B5D5B4AC1E5F4C338DE3BAB830B4BAEF
2016-01-14 18:31:33,827 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_-1"]
2016-01-14 18:31:33,827 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 579819F5447A4008A0635900C96279B9, for 1 inputs
2016-01-14 18:31:33,827 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C5C25D56F95D4CD2A19A23083E22D9E7, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:33,827 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2b8d94aa; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5c1b3b3c
2016-01-14 18:31:33,828 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:33,828 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:33,828 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:33,829 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:33,829 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796293555_0001_1_01_000000_0
2016-01-14 18:31:33,829 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796293555_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:33,830 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796293555_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:33,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796293827, Process_Duration=2, Process_End_Time=1452796293829, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:33,830 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C5C25D56F95D4CD2A19A23083E22D9E7, taskAttemptId=attempt_1452796293555_0001_1_01_000000_0, startTime=1452796293795, finishTime=1452796293830, timeTaken=35, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=10, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796293827, Process_Duration=2, Process_End_Time=1452796293829, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:33,831 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796293555_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:33,831 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796293555_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:33,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:TASK_FINISHED]: vertexName=C5C25D56F95D4CD2A19A23083E22D9E7, taskId=task_1452796293555_0001_1_01_000000, startTime=1452796293795, finishTime=1452796293831, timeTaken=36, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:33,831 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796293555_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:33,831 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] : 1
2016-01-14 18:31:33,831 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] because task failed: task_1452796293555_0001_1_01_000000
2016-01-14 18:31:33,832 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:33,832 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:33,832 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7]
2016-01-14 18:31:33,832 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=B5D5B4AC1E5F4C338DE3BAB830B4BAEF, vertexId=vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7]
2016-01-14 18:31:33,831 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C5C25D56F95D4CD2A19A23083E22D9E7, mem on close (mb), free: 119, total: 371, max: 672
2016-01-14 18:31:33,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=C5C25D56F95D4CD2A19A23083E22D9E7, index=1
2016-01-14 18:31:33,832 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D5E03B5D77A84DCA8F0AB28AFB1369F3
2016-01-14 18:31:33,833 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:33,833 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:33,833 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:33,833 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:33,834 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:33,834 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:33,834 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:33,835 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:VERTEX_FINISHED]: vertexName=C5C25D56F95D4CD2A19A23083E22D9E7, vertexId=vertex_1452796293555_0001_1_01, initRequestedTime=1452796293715, initedTime=1452796293721, startRequestedTime=1452796293732, startedTime=1452796293732, finishTime=1452796293835, timeTaken=103, status=FAILED, diagnostics=Task failed, taskId=task_1452796293555_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796293795, firstTasksToStart=[ task_1452796293555_0001_1_01_000000 ], lastTaskFinishTime=1452796293830, lastTasksToFinish=[ task_1452796293555_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:33,835 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:33,835 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:33,835 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:33,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=C5C25D56F95D4CD2A19A23083E22D9E7, sourceVertex=D5E03B5D77A84DCA8F0AB28AFB1369F3
2016-01-14 18:31:33,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=C5C25D56F95D4CD2A19A23083E22D9E7, sourceVertex=B5D5B4AC1E5F4C338DE3BAB830B4BAEF
2016-01-14 18:31:33,837 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796293555_0001_00_000002, stoppedTime=1452796293837, exitStatus=0
2016-01-14 18:31:33,837 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:33,837 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796293555_0001_1_01_000000_0 failed: true container: container_1452796293555_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:33,837 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:33,838 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:33,838 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796293555_0001_1
2016-01-14 18:31:33,838 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: B5D5B4AC1E5F4C338DE3BAB830B4BAEF for vertex: vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7]
2016-01-14 18:31:33,837 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796293555_0001_00_000002
2016-01-14 18:31:33,838 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796293555_0001_1_00 [D5E03B5D77A84DCA8F0AB28AFB1369F3]
2016-01-14 18:31:33,839 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796293555_0001_1, queueSize=0
2016-01-14 18:31:33,839 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796293555_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796293555_0001_1, startTime=1452796293713, finishTime=1452796293838, timeTaken=125, status=FAILED, diagnostics=Vertex failed, vertexName=C5C25D56F95D4CD2A19A23083E22D9E7, vertexId=vertex_1452796293555_0001_1_01, diagnostics=[Task failed, taskId=task_1452796293555_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796293555_0001_1_01 [C5C25D56F95D4CD2A19A23083E22D9E7] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796293775, Process_Duration=2, Process_End_Time=1452796293777, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:33,839 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796293555_0001_1 finished with state: FAILED
2016-01-14 18:31:33,839 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796293555_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:33,837 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:33,838 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796293555_0001_00_000002
2016-01-14 18:31:33,840 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:33,840 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796293555_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:33,840 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796293555_0001_00_000002 completed but with errors
2016-01-14 18:31:33,841 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796293555_0001_00_000002
2016-01-14 18:31:33 Completed Dag: dag_1452796293555_0001_1
2016-01-14 18:31:33,841 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796293555_0001_1, dagState=FAILED
2016-01-14 18:31:33,841 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796293555_0001_1_post
2016-01-14 18:31:33,841 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:33,841 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:33,841 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796293555_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:33,841 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796293555_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:33,841 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796293555_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:33,851 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/0_VALUES_-1, applicationId=application_1452796293555_0001
2016-01-14 18:31:33,853 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:33,853 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:33,853 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:33,853 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:33,853 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:33,854 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:33,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:33,854 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:33,854 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:33,854 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:33,854 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:33,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:33,855 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:33,860 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C64406301BFC4FE9ACD3FCF0D341DF5A/FAC7ED434789424CB0127408C3CD7BBA/.tez/application_1452796293555_0001
2016-01-14 18:31:33,860 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:33,860 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...emblyplatform/0_VALUES_-1
2016-01-14 18:31:33,861 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:33,861 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.324
2016-01-14 18:31:33,861 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_-1
2016-01-14 18:31:33,883 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:33,883 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:33,923 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-01-14 18:31:33,925 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-01-14 18:31:33,925 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:33,925 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:33,925 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_'field'
2016-01-14 18:31:33,946 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:33,946 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:33,975 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 18:31:33,980 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 18:31:33,980 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:33,981 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:33,981 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:33,995 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:34,002 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS
2016-01-14 18:31:34,047 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:34,047 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:34,047 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS"]
2016-01-14 18:31:34,047 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:34,047 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:34,048 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:34,048 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/0_VALUES_RESULTS
2016-01-14 18:31:34,048 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:34,061 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:34,065 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:34,065 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:34,065 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:34,074 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B/.tez/application_1452796294065_0001 doesn't exist and is created
2016-01-14 18:31:34,087 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:34,087 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:34,087 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B/.tez/application_1452796294065_0001
2016-01-14 18:31:34,089 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796294065_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:34,099 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:34,099 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:34,101 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:34,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:34,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B/.tez/application_1452796294065_0001/localmode-log-dir
2016-01-14 18:31:34,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B/.tez/application_1452796294065_0001/localmode-log-dir/history.txt.appattempt_1452796294065_0001_000000, maxErrors=10
2016-01-14 18:31:34,101 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:34,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796294065_0001_000000, appSubmitTime=1452796294089, launchTime=1452796294089
2016-01-14 18:31:34,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:34,106 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59156
2016-01-14 18:31:34,109 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:34,110 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:34,111 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:34,112 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:34,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796294065_0001_000000, startTime=1452796294115
2016-01-14 18:31:34,115 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:34,115 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:34,187 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:34,187 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:34,187 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1452796294065_0001, dagName=[DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B] every/(1/1) ...platform/0_VALUES_RESULTS
2016-01-14 18:31:34,203 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B] every/(1/1) ...platform/0_VALUES_RESULTS
2016-01-14 18:31:34,204 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796294065_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B/.tez/application_1452796294065_0001/localmode-log-dir/dag_1452796294065_0001_1.dot
2016-01-14 18:31:34,205 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B/.tez/application_1452796294065_0001/localmode-log-dir/dag_1452796294065_0001_1-tez-dag.pb.txt
2016-01-14 18:31:34,211 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796294065_0001_1
2016-01-14 18:31:34,212 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:34,212 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B] every/(1/1) ...platform/0_VALUES_RESULTS
2016-01-14 18:31:34 Running Dag: dag_1452796294065_0001_1
2016-01-14 18:31:34,221 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796294065_0001_1, submitTime=1452796294203
2016-01-14 18:31:34,221 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:34,221 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A2B7E4000C154FDAB7E6EAF0093479A1
2016-01-14 18:31:34,222 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:34,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796294065_0001_1, initTime=1452796294221
2016-01-14 18:31:34,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:34,222 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796294065_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:34,222 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:DAG_STARTED]: dagID=dag_1452796294065_0001_1, startTime=1452796294222
2016-01-14 18:31:34,222 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:34,222 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:34,223 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796294065_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:34,223 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B8951AF6897046EBB7D899B7CD63FB98 : {9392A58384AF44AEB884A01930E7D608={InputName=9392A58384AF44AEB884A01930E7D608}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:34,223 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 9392A58384AF44AEB884A01930E7D608, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:34,223 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98]
2016-01-14 18:31:34,223 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98]
2016-01-14 18:31:34,223 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98]
2016-01-14 18:31:34,223 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98]. Starting root input initializers: 1
2016-01-14 18:31:34,224 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:34,224 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1]
2016-01-14 18:31:34,224 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1452796294065_0001, dagName=[DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B] every/(1/1) ...platform/0_VALUES_RESULTS
2016-01-14 18:31:34,225 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 9392A58384AF44AEB884A01930E7D608 on vertex vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98]
2016-01-14 18:31:34,227 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:34,227 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:34,227 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 9392A58384AF44AEB884A01930E7D608 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:34,227 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1]
2016-01-14 18:31:34,228 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:34,228 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1]
2016-01-14 18:31:34,228 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:34,228 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1]
2016-01-14 18:31:34,229 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A4A9FA8750D34C648E127EBCBC859441, vertexId=vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:34,229 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A4A9FA8750D34C648E127EBCBC859441, vertexId=vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1]
2016-01-14 18:31:34,230 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:34,230 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 9392A58384AF44AEB884A01930E7D608 on vertex vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98]
2016-01-14 18:31:34,231 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A4A9FA8750D34C648E127EBCBC859441, vertexName=A2B7E4000C154FDAB7E6EAF0093479A1, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:34,231 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A4A9FA8750D34C648E127EBCBC859441, vertexId=vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1]
2016-01-14 18:31:34,232 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:34,232 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A2B7E4000C154FDAB7E6EAF0093479A1, vertexId=vertex_1452796294065_0001_1_01, initRequestedTime=1452796294224, initedTime=1452796294232, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:34,233 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:34,233 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:34,233 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {9392A58384AF44AEB884A01930E7D608=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:34,233 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98] parallelism set to 1
2016-01-14 18:31:34,234 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:34,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796294065_0001_1_00, eventCount=1
2016-01-14 18:31:34,234 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:34,235 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B8951AF6897046EBB7D899B7CD63FB98, vertexId=vertex_1452796294065_0001_1_00, initRequestedTime=1452796294223, initedTime=1452796294234, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:34,235 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98] with distanceFromRoot: 0
2016-01-14 18:31:34,235 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:34,235 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B8951AF6897046EBB7D899B7CD63FB98
2016-01-14 18:31:34,235 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98]
2016-01-14 18:31:34,236 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:34,236 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796294065_0001_1_00, startRequestedTime=1452796294235, startedTime=1452796294235
2016-01-14 18:31:34,236 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:34,237 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:34,237 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:TASK_STARTED]: vertexName=B8951AF6897046EBB7D899B7CD63FB98, taskId=task_1452796294065_0001_1_00_000000, scheduledTime=1452796294237, launchTime=1452796294237
2016-01-14 18:31:34,237 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294065_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:34,237 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796294065_0001_1_00 for vertex: vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] numStartedSources: 1 numSources: 1
2016-01-14 18:31:34,238 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] with distanceFromRoot: 1
2016-01-14 18:31:34,238 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98]
2016-01-14 18:31:34,238 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B8951AF6897046EBB7D899B7CD63FB98 in vertex: A2B7E4000C154FDAB7E6EAF0093479A1
2016-01-14 18:31:34,239 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A2B7E4000C154FDAB7E6EAF0093479A1 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:34,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:34,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796294065_0001_1_01, startRequestedTime=1452796294238, startedTime=1452796294238
2016-01-14 18:31:34,239 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:34,240 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: B8951AF6897046EBB7D899B7CD63FB98, VertexParallelism: 1, TaskAttemptID:attempt_1452796294065_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9392A58384AF44AEB884A01930E7D608, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A2B7E4000C154FDAB7E6EAF0093479A1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:34,240 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294065_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:34,240 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:34,241 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:34,241 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:34,241 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:34,242 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796294065_0001_00_000001
2016-01-14 18:31:34,242 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:34,244 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796294065_0001_00_000001
2016-01-14 18:31:34,244 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:34,245 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796294065_0001, with dag id: dag_1452796294065_0001_1
2016-01-14 18:31:34,245 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:34,245 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:34,245 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796294065_0001_00_000001, launchTime=1452796294244
2016-01-14 18:31:34,246 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796294065_0001_00_000001
2016-01-14 18:31:34,247 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796294065_0001_1_00_000000_0] to container: [container_1452796294065_0001_00_000001]
2016-01-14 18:31:34,247 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:34,247 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796294065_0001_00_000001 given task: attempt_1452796294065_0001_1_00_000000_0
2016-01-14 18:31:34,247 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796294065_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796294065_0001_1_00_000000_0
2016-01-14 18:31:34 Starting to run new task attempt: attempt_1452796294065_0001_1_00_000000_0
2016-01-14 18:31:34,248 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796294065_0001_1_00_000000_0] started. Is using containerId: [container_1452796294065_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:34,248 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796294065_0001_1_00_000000_0
2016-01-14 18:31:34,248 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:34,248 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:34,248 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:34,248 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:34,248 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B8951AF6897046EBB7D899B7CD63FB98, taskAttemptId=attempt_1452796294065_0001_1_00_000000_0, startTime=1452796294247, containerId=container_1452796294065_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796294065_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:34,248 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:34,248 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:34,248 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:34,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: B8951AF6897046EBB7D899B7CD63FB98, VertexParallelism: 1, TaskAttemptID:attempt_1452796294065_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=9392A58384AF44AEB884A01930E7D608, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A2B7E4000C154FDAB7E6EAF0093479A1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:34,249 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:34,249 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294065_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:34,249 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294065_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:34,250 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796294065_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:34,250 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796294065_0001_1_00_000000_0
2016-01-14 18:31:34,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:34,250 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=9392A58384AF44AEB884A01930E7D608, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:34,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:34,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 9392A58384AF44AEB884A01930E7D608
2016-01-14 18:31:34,252 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 9392A58384AF44AEB884A01930E7D608
2016-01-14 18:31:34,253 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:34,254 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:34,254 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 9392A58384AF44AEB884A01930E7D608
2016-01-14 18:31:34,254 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 9392A58384AF44AEB884A01930E7D608
2016-01-14 18:31:34,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A2B7E4000C154FDAB7E6EAF0093479A1, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:34,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:34,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A2B7E4000C154FDAB7E6EAF0093479A1
2016-01-14 18:31:34,256 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B8951AF6897046EBB7D899B7CD63FB98, ordinal: 0
2016-01-14 18:31:34,256 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B8951AF6897046EBB7D899B7CD63FB98, mem on start (mb), free: 221, total: 371, max: 672
2016-01-14 18:31:34,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:34,257 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:34,258 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:34,259 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A2B7E4000C154FDAB7E6EAF0093479A1
2016-01-14 18:31:34,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A2B7E4000C154FDAB7E6EAF0093479A1
2016-01-14 18:31:34,259 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:34,260 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:34,260 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:34,260 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:34,260 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:34,260 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:34,260 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:34,261 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 9392A58384AF44AEB884A01930E7D608, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:34,261 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A2B7E4000C154FDAB7E6EAF0093479A1, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:34,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:34,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 9392A58384AF44AEB884A01930E7D608 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:34,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:34,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:34,261 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 9392A58384AF44AEB884A01930E7D608
2016-01-14 18:31:34,262 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:34,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 9392A58384AF44AEB884A01930E7D608
2016-01-14 18:31:34,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:34,262 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796294065_0001_1_00_000000_0
2016-01-14 18:31:34,262 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:34,263 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@75e8fc6
2016-01-14 18:31:34,264 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@75e8fc6
2016-01-14 18:31:34,264 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:34,264 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:34,266 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:34,266 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 9392A58384AF44AEB884A01930E7D608
2016-01-14 18:31:34,266 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 349121B35A27490CB387C178717D2384
2016-01-14 18:31:34,266 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 349121B35A27490CB387C178717D2384
2016-01-14 18:31:34,267 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@4c1883a8; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5d6bb19f; comparator=cascading.tuple.hadoop.util.TupleComparator@495a6da3; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:34,267 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:34,267 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:34,267 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:34,280 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:34,280 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:34,280 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:34,280 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:34,280 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:34,281 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:34,281 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:34,281 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B8951AF6897046EBB7D899B7CD63FB98, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:34,284 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796294065_0001_1_00_000000_0
2016-01-14 18:31:34,284 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B8951AF6897046EBB7D899B7CD63FB98, mem on close (mb), free: 117, total: 371, max: 672
2016-01-14 18:31:34,284 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:34,284 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:34,284 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:34,285 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:34,286 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:34 Completed running task attempt: attempt_1452796294065_0001_1_00_000000_0
2016-01-14 18:31:34,288 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796294065_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:34,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796294065_0001_1_00, eventCount=1
2016-01-14 18:31:34,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796294281, Process_Duration=3, Process_End_Time=1452796294284, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:34,289 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:34,289 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:34,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B8951AF6897046EBB7D899B7CD63FB98, taskAttemptId=attempt_1452796294065_0001_1_00_000000_0, startTime=1452796294247, finishTime=1452796294289, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796294281, Process_Duration=3, Process_End_Time=1452796294284, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:34,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:34,289 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294065_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:34,289 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:34,289 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796294065_0001_1_00_000000_0
2016-01-14 18:31:34,289 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796294065_0001_00_000001
2016-01-14 18:31:34,291 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:34,291 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:TASK_FINISHED]: vertexName=B8951AF6897046EBB7D899B7CD63FB98, taskId=task_1452796294065_0001_1_00_000000, startTime=1452796294247, finishTime=1452796294289, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1452796294065_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796294281, Process_Duration=3, Process_End_Time=1452796294284, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:34,291 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294065_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:34,292 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98] : 1
2016-01-14 18:31:34,292 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98]
2016-01-14 18:31:34,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:34,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:VERTEX_FINISHED]: vertexName=B8951AF6897046EBB7D899B7CD63FB98, vertexId=vertex_1452796294065_0001_1_00, initRequestedTime=1452796294223, initedTime=1452796294234, startRequestedTime=1452796294235, startedTime=1452796294235, finishTime=1452796294292, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796294281, Process_Duration=3, Process_End_Time=1452796294284, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796294247, firstTasksToStart=[ task_1452796294065_0001_1_00_000000 ], lastTaskFinishTime=1452796294289, lastTasksToFinish=[ task_1452796294065_0001_1_00_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796294065_0001_1_00_000000 ], longestDurationTasks=[ task_1452796294065_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:34,292 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:34,292 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:34,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796294065_0001_00_000001, stoppedTime=1452796294292, exitStatus=0
2016-01-14 18:31:34,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:34,293 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:34,293 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] attempt: attempt_1452796294065_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:34,293 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A2B7E4000C154FDAB7E6EAF0093479A1
2016-01-14 18:31:34,293 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1]
2016-01-14 18:31:34,293 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:34,293 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796294065_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:34,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:TASK_STARTED]: vertexName=A2B7E4000C154FDAB7E6EAF0093479A1, taskId=task_1452796294065_0001_1_01_000000, scheduledTime=1452796294293, launchTime=1452796294293
2016-01-14 18:31:34,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:34,293 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294065_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:34,293 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1]
2016-01-14 18:31:34,294 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:34,294 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: A2B7E4000C154FDAB7E6EAF0093479A1, VertexParallelism: 1, TaskAttemptID:attempt_1452796294065_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B8951AF6897046EBB7D899B7CD63FB98, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A4A9FA8750D34C648E127EBCBC859441, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:34,294 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294065_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:34,294 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:34,294 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796294065_0001_00_000002
2016-01-14 18:31:34,294 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:34,296 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796294065_0001_00_000002
2016-01-14 18:31:34,296 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:34,299 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:34,299 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:34,299 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796294065_0001_00_000002, launchTime=1452796294299
2016-01-14 18:31:34,300 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796294065_0001_00_000002
2016-01-14 18:31:34,300 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796294065_0001_1_01_000000_0] to container: [container_1452796294065_0001_00_000002]
2016-01-14 18:31:34,300 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:34,301 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796294065_0001_00_000002 given task: attempt_1452796294065_0001_1_01_000000_0
2016-01-14 18:31:34,301 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796294065_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796294065_0001_1_01_000000_0
2016-01-14 18:31:34,301 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796294065_0001_1_01_000000_0] started. Is using containerId: [container_1452796294065_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:34,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A2B7E4000C154FDAB7E6EAF0093479A1, taskAttemptId=attempt_1452796294065_0001_1_01_000000_0, startTime=1452796294301, containerId=container_1452796294065_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796294065_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:34,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:34 Starting to run new task attempt: attempt_1452796294065_0001_1_01_000000_0
2016-01-14 18:31:34,301 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294065_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:34,301 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796294065_0001_1_01_000000_0
2016-01-14 18:31:34,301 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:34,302 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796294065_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:34,302 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796294065_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:34,301 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294065_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:34,302 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796294065_0001_00_000001, Exiting
2016-01-14 18:31:34,302 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796294065_0001_00_000001
2016-01-14 18:31:34,302 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796294065_0001_00_000001
2016-01-14 18:31:34,302 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:34,302 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:34,302 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:34,303 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:34,303 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:34,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B] every/(1/1) ...platform/0_VALUES_RESULTS, VertexName: A2B7E4000C154FDAB7E6EAF0093479A1, VertexParallelism: 1, TaskAttemptID:attempt_1452796294065_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B8951AF6897046EBB7D899B7CD63FB98, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A4A9FA8750D34C648E127EBCBC859441, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:34,303 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:34,303 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796294065_0001_1_01_000000_0
2016-01-14 18:31:34,303 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796294065_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:34,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:34,304 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,304 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796294065_0001_00_000001 completed successfully
2016-01-14 18:31:34,304 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:34,304 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796294065_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:34,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B8951AF6897046EBB7D899B7CD63FB98, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:34,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:34,305 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B8951AF6897046EBB7D899B7CD63FB98
2016-01-14 18:31:34,307 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:34,307 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:34,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A4A9FA8750D34C648E127EBCBC859441, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:34,307 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:34,308 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A4A9FA8750D34C648E127EBCBC859441
2016-01-14 18:31:34,308 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:34,309 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A4A9FA8750D34C648E127EBCBC859441
2016-01-14 18:31:34,311 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:34,311 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B8951AF6897046EBB7D899B7CD63FB98
2016-01-14 18:31:34,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B8951AF6897046EBB7D899B7CD63FB98
2016-01-14 18:31:34,313 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A2B7E4000C154FDAB7E6EAF0093479A1, ordinal: 1
2016-01-14 18:31:34,313 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2B7E4000C154FDAB7E6EAF0093479A1, mem on start (mb), free: 110, total: 371, max: 672
2016-01-14 18:31:34,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:34,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:34,316 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:34,316 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A4A9FA8750D34C648E127EBCBC859441
2016-01-14 18:31:34,316 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:34,316 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:34,316 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:34,317 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:34,317 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:34,317 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:34,317 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:34,317 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A4A9FA8750D34C648E127EBCBC859441, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:34,317 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B8951AF6897046EBB7D899B7CD63FB98, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:34,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:34,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B8951AF6897046EBB7D899B7CD63FB98 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:34,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:34,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:34,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B8951AF6897046EBB7D899B7CD63FB98
2016-01-14 18:31:34,319 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:34,319 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:34,320 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B8951AF6897046EBB7D899B7CD63FB98 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:34,320 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:34,320 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:34,320 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:34,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B8951AF6897046EBB7D899B7CD63FB98
2016-01-14 18:31:34,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:34,322 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796294065_0001_1_01_000000_0
2016-01-14 18:31:34,323 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796294065_0001_1_00_000000_0_10183, runDuration: 30980, ]
2016-01-14 18:31:34,323 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796294065_0001_1_00_000000_0_10183] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796294323, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:34,324 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B8951AF6897046EBB7D899B7CD63FB98
2016-01-14 18:31:34,324 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:34,324 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B8951AF6897046EBB7D899B7CD63FB98] #31 in 1ms
2016-01-14 18:31:34,324 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B8951AF6897046EBB7D899B7CD63FB98] #31, status:false, isInterrupted:false
2016-01-14 18:31:34,324 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:34,324 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:34,324 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:34,324 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:34,325 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:34,325 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B8951AF6897046EBB7D899B7CD63FB98
2016-01-14 18:31:34,325 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:34,326 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:34,326 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 349121B35A27490CB387C178717D2384
2016-01-14 18:31:34,326 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS"], id: A4A9FA8750D34C648E127EBCBC859441
2016-01-14 18:31:34,327 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_RESULTS"]
2016-01-14 18:31:34,327 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 349121B35A27490CB387C178717D2384, for 1 inputs
2016-01-14 18:31:34,327 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A2B7E4000C154FDAB7E6EAF0093479A1, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:34,327 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3559d847; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1d8638ae
2016-01-14 18:31:34,328 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:34,329 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:34,329 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:34,329 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:34,329 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796294065_0001_1_01_000000_0
2016-01-14 18:31:34,329 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796294065_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:34,330 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796294065_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:34,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A2B7E4000C154FDAB7E6EAF0093479A1, taskAttemptId=attempt_1452796294065_0001_1_01_000000_0, startTime=1452796294301, finishTime=1452796294331, timeTaken=30, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796294327, Process_Duration=3, Process_End_Time=1452796294330, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:34,331 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294065_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:34,331 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796294065_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:34,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:TASK_FINISHED]: vertexName=A2B7E4000C154FDAB7E6EAF0093479A1, taskId=task_1452796294065_0001_1_01_000000, startTime=1452796294301, finishTime=1452796294331, timeTaken=30, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:34,332 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294065_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:34,332 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] : 1
2016-01-14 18:31:34,332 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:34,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:34,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:34,332 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] because task failed: task_1452796294065_0001_1_01_000000
2016-01-14 18:31:34,333 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:34,333 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:34,333 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1]
2016-01-14 18:31:34,333 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=A4A9FA8750D34C648E127EBCBC859441, vertexId=vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1]
2016-01-14 18:31:34,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:VERTEX_FINISHED]: vertexName=A2B7E4000C154FDAB7E6EAF0093479A1, vertexId=vertex_1452796294065_0001_1_01, initRequestedTime=1452796294224, initedTime=1452796294232, startRequestedTime=1452796294238, startedTime=1452796294238, finishTime=1452796294334, timeTaken=96, status=FAILED, diagnostics=Task failed, taskId=task_1452796294065_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796294301, firstTasksToStart=[ task_1452796294065_0001_1_01_000000 ], lastTaskFinishTime=1452796294331, lastTasksToFinish=[ task_1452796294065_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:34,335 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:34,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:34,335 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796294065_0001_00_000002, stoppedTime=1452796294335, exitStatus=0
2016-01-14 18:31:34,336 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:34,336 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796294065_0001_1_01_000000_0 failed: true container: container_1452796294065_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:34,336 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:34,336 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:34,336 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796294065_0001_1
2016-01-14 18:31:34,336 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: A4A9FA8750D34C648E127EBCBC859441 for vertex: vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1]
2016-01-14 18:31:34,336 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796294065_0001_1_00 [B8951AF6897046EBB7D899B7CD63FB98]
2016-01-14 18:31:34,337 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:34,338 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796294065_0001_1, queueSize=0
2016-01-14 18:31:34,338 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:34,340 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294065_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796294065_0001_1, startTime=1452796294222, finishTime=1452796294336, timeTaken=114, status=FAILED, diagnostics=Vertex failed, vertexName=A2B7E4000C154FDAB7E6EAF0093479A1, vertexId=vertex_1452796294065_0001_1_01, diagnostics=[Task failed, taskId=task_1452796294065_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796294065_0001_1_01 [A2B7E4000C154FDAB7E6EAF0093479A1] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796294281, Process_Duration=3, Process_End_Time=1452796294284, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:34,340 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796294065_0001_1 finished with state: FAILED
2016-01-14 18:31:34,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=389545984, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796294327, Process_Duration=3, Process_End_Time=1452796294330, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:34,340 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A2B7E4000C154FDAB7E6EAF0093479A1, mem on close (mb), free: 103, total: 371, max: 672
2016-01-14 18:31:34,340 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796294065_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:34,340 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=A2B7E4000C154FDAB7E6EAF0093479A1, index=1
2016-01-14 18:31:34,341 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B8951AF6897046EBB7D899B7CD63FB98
2016-01-14 18:31:34,341 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:34,341 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:34,341 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:34,342 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796294065_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:34,342 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:34,342 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:34,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=A2B7E4000C154FDAB7E6EAF0093479A1, sourceVertex=B8951AF6897046EBB7D899B7CD63FB98
2016-01-14 18:31:34,342 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=A2B7E4000C154FDAB7E6EAF0093479A1, sourceVertex=A4A9FA8750D34C648E127EBCBC859441
2016-01-14 18:31:34,343 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796294065_0001_00_000002
2016-01-14 18:31:34,343 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796294065_0001_00_000002
2016-01-14 18:31:34,343 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796294065_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:34,343 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796294065_0001_00_000002 completed but with errors
2016-01-14 18:31:34 Completed Dag: dag_1452796294065_0001_1
2016-01-14 18:31:34,344 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796294065_0001_1, dagState=FAILED
2016-01-14 18:31:34,345 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796294065_0001_1_post
2016-01-14 18:31:34,345 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:34,345 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:34,345 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:34,347 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796294065_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:34,347 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294065_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:34,347 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294065_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:34,347 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/0_VALUES_RESULTS, applicationId=application_1452796294065_0001
2016-01-14 18:31:34,348 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:34,348 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:34,348 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:34,348 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:34,349 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:34,349 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:34,349 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:34,349 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:34,349 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:34,350 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:34,350 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:34,350 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:34,350 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:34,355 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/DA5A75B523E84B80A1644F5F17A0889F/DD495341B3F8400F86B50A4E73D47A3B/.tez/application_1452796294065_0001
2016-01-14 18:31:34,356 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:34,356 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...platform/0_VALUES_RESULTS
2016-01-14 18:31:34,356 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:34,357 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.309
2016-01-14 18:31:34,357 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_RESULTS
2016-01-14 18:31:34,375 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:34,375 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:34,392 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:34,392 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:34,435 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 18:31:34,447 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.023
2016-01-14 18:31:34,448 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:34,448 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:34,448 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:34,470 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:34,477 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL
2016-01-14 18:31:34,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:34,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:34,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL"]
2016-01-14 18:31:34,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:34,547 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:34,548 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:34,548 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 18:31:34,548 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:34,567 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:34,571 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:34,572 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:34,572 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:34,581 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C/.tez/application_1452796294571_0001 doesn't exist and is created
2016-01-14 18:31:34,597 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:34,598 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:34,598 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C/.tez/application_1452796294571_0001
2016-01-14 18:31:34,600 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796294571_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:34,616 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:34,617 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:34,618 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:34,619 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:34,619 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C/.tez/application_1452796294571_0001/localmode-log-dir
2016-01-14 18:31:34,619 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C/.tez/application_1452796294571_0001/localmode-log-dir/history.txt.appattempt_1452796294571_0001_000000, maxErrors=10
2016-01-14 18:31:34,619 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:34,619 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796294571_0001_000000, appSubmitTime=1452796294600, launchTime=1452796294600
2016-01-14 18:31:34,621 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:34,623 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:34,626 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:34,626 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:34,629 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:42190
2016-01-14 18:31:34,631 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:34,632 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796294571_0001_000000, startTime=1452796294632
2016-01-14 18:31:34,632 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:34,632 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:34,698 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:34,698 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:34,698 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1452796294571_0001, dagName=[D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 18:31:34,714 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 18:31:34,715 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796294571_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C/.tez/application_1452796294571_0001/localmode-log-dir/dag_1452796294571_0001_1.dot
2016-01-14 18:31:34,715 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C/.tez/application_1452796294571_0001/localmode-log-dir/dag_1452796294571_0001_1-tez-dag.pb.txt
2016-01-14 18:31:34,722 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796294571_0001_1
2016-01-14 18:31:34,722 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:34,722 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 18:31:34 Running Dag: dag_1452796294571_0001_1
2016-01-14 18:31:34,731 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796294571_0001_1, submitTime=1452796294714
2016-01-14 18:31:34,731 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:34,731 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F4056EEC45C84599BC7AEDF9DFD1056B
2016-01-14 18:31:34,731 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:34,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796294571_0001_1, initTime=1452796294731
2016-01-14 18:31:34,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:34,732 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796294571_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:34,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:DAG_STARTED]: dagID=dag_1452796294571_0001_1, startTime=1452796294732
2016-01-14 18:31:34,732 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:34,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:34,733 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796294571_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:34,733 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E263AFBD7D8741639EEA6F9E113F2556 : {FE9C40F6B0A14603B0CEA2EBF1BCC907={InputName=FE9C40F6B0A14603B0CEA2EBF1BCC907}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:34,733 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: FE9C40F6B0A14603B0CEA2EBF1BCC907, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:34,733 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556]
2016-01-14 18:31:34,733 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556]
2016-01-14 18:31:34,733 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556]
2016-01-14 18:31:34,733 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556]. Starting root input initializers: 1
2016-01-14 18:31:34,734 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:34,734 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B]
2016-01-14 18:31:34,734 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: FE9C40F6B0A14603B0CEA2EBF1BCC907 on vertex vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556]
2016-01-14 18:31:34,735 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1452796294571_0001, dagName=[D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C] every/(1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 18:31:34,735 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:34,736 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input FE9C40F6B0A14603B0CEA2EBF1BCC907 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:34,736 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:34,736 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:34,737 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:34,737 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B]
2016-01-14 18:31:34,737 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B]
2016-01-14 18:31:34,737 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B]
2016-01-14 18:31:34,737 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1909503A65EB4B89B03FF53A9AE6A9E9, vertexId=vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:34,737 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1909503A65EB4B89B03FF53A9AE6A9E9, vertexId=vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B]
2016-01-14 18:31:34,738 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:34,738 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: FE9C40F6B0A14603B0CEA2EBF1BCC907 on vertex vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556]
2016-01-14 18:31:34,739 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1909503A65EB4B89B03FF53A9AE6A9E9, vertexName=F4056EEC45C84599BC7AEDF9DFD1056B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:34,739 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1909503A65EB4B89B03FF53A9AE6A9E9, vertexId=vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B]
2016-01-14 18:31:34,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F4056EEC45C84599BC7AEDF9DFD1056B, vertexId=vertex_1452796294571_0001_1_01, initRequestedTime=1452796294734, initedTime=1452796294740, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:34,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:34,740 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:34,740 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:34,740 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {FE9C40F6B0A14603B0CEA2EBF1BCC907=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:34,741 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556] parallelism set to 1
2016-01-14 18:31:34,741 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:34,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796294571_0001_1_00, eventCount=1
2016-01-14 18:31:34,742 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E263AFBD7D8741639EEA6F9E113F2556, vertexId=vertex_1452796294571_0001_1_00, initRequestedTime=1452796294733, initedTime=1452796294741, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:34,742 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556] with distanceFromRoot: 0
2016-01-14 18:31:34,742 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:34,742 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:34,742 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E263AFBD7D8741639EEA6F9E113F2556
2016-01-14 18:31:34,742 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556]
2016-01-14 18:31:34,743 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796294571_0001_1_00, startRequestedTime=1452796294742, startedTime=1452796294742
2016-01-14 18:31:34,743 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:34,743 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:34,744 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:34,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:TASK_STARTED]: vertexName=E263AFBD7D8741639EEA6F9E113F2556, taskId=task_1452796294571_0001_1_00_000000, scheduledTime=1452796294743, launchTime=1452796294743
2016-01-14 18:31:34,744 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294571_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:34,744 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796294571_0001_1_00 for vertex: vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] numStartedSources: 1 numSources: 1
2016-01-14 18:31:34,744 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] with distanceFromRoot: 1
2016-01-14 18:31:34,744 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556]
2016-01-14 18:31:34,745 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E263AFBD7D8741639EEA6F9E113F2556 in vertex: F4056EEC45C84599BC7AEDF9DFD1056B
2016-01-14 18:31:34,745 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F4056EEC45C84599BC7AEDF9DFD1056B with 1 source tasks and 1 pending tasks
2016-01-14 18:31:34,745 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796294571_0001_1_01, startRequestedTime=1452796294745, startedTime=1452796294745
2016-01-14 18:31:34,745 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:34,745 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:34,746 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: E263AFBD7D8741639EEA6F9E113F2556, VertexParallelism: 1, TaskAttemptID:attempt_1452796294571_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FE9C40F6B0A14603B0CEA2EBF1BCC907, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F4056EEC45C84599BC7AEDF9DFD1056B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:34,746 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294571_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:34,746 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:34,747 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:34,747 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:34,747 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:34,748 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796294571_0001_00_000001
2016-01-14 18:31:34,748 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:34,749 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796294571_0001_00_000001
2016-01-14 18:31:34,749 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:34,749 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:34,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:34,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796294571_0001_00_000001, launchTime=1452796294750
2016-01-14 18:31:34,750 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796294571_0001_00_000001
2016-01-14 18:31:34,751 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796294571_0001_1_00_000000_0] to container: [container_1452796294571_0001_00_000001]
2016-01-14 18:31:34,751 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:34,751 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796294571_0001_00_000001 given task: attempt_1452796294571_0001_1_00_000000_0
2016-01-14 18:31:34,751 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796294571_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796294571_0001_1_00_000000_0
2016-01-14 18:31:34,751 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796294571_0001_1_00_000000_0] started. Is using containerId: [container_1452796294571_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:34,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E263AFBD7D8741639EEA6F9E113F2556, taskAttemptId=attempt_1452796294571_0001_1_00_000000_0, startTime=1452796294751, containerId=container_1452796294571_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796294571_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:34,752 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294571_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:34 Starting to run new task attempt: attempt_1452796294571_0001_1_00_000000_0
2016-01-14 18:31:34,752 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294571_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:34,752 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796294571_0001_1_00_000000_0
2016-01-14 18:31:34,752 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:34,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:34,752 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:34,752 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:34,752 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:34,753 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:34,753 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796294571_0001, with dag id: dag_1452796294571_0001_1
2016-01-14 18:31:34,753 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:34,754 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: E263AFBD7D8741639EEA6F9E113F2556, VertexParallelism: 1, TaskAttemptID:attempt_1452796294571_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FE9C40F6B0A14603B0CEA2EBF1BCC907, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F4056EEC45C84599BC7AEDF9DFD1056B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:34,754 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:34,755 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796294571_0001_1_00_000000_0
2016-01-14 18:31:34,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:34,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,755 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796294571_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:34,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FE9C40F6B0A14603B0CEA2EBF1BCC907, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:34,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:34,756 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FE9C40F6B0A14603B0CEA2EBF1BCC907
2016-01-14 18:31:34,757 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: FE9C40F6B0A14603B0CEA2EBF1BCC907
2016-01-14 18:31:34,758 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:34,761 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:34,761 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E263AFBD7D8741639EEA6F9E113F2556, ordinal: 0
2016-01-14 18:31:34,761 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E263AFBD7D8741639EEA6F9E113F2556, mem on start (mb), free: 226, total: 372, max: 672
2016-01-14 18:31:34,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:34,761 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: FE9C40F6B0A14603B0CEA2EBF1BCC907
2016-01-14 18:31:34,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FE9C40F6B0A14603B0CEA2EBF1BCC907
2016-01-14 18:31:34,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:34,761 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F4056EEC45C84599BC7AEDF9DFD1056B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:34,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:34,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F4056EEC45C84599BC7AEDF9DFD1056B
2016-01-14 18:31:34,763 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:34,763 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F4056EEC45C84599BC7AEDF9DFD1056B
2016-01-14 18:31:34,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F4056EEC45C84599BC7AEDF9DFD1056B
2016-01-14 18:31:34,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:34,763 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:34,764 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:34,764 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:34,764 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:34,764 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:34,764 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:34,764 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FE9C40F6B0A14603B0CEA2EBF1BCC907, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:34,765 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F4056EEC45C84599BC7AEDF9DFD1056B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:34,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:34,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FE9C40F6B0A14603B0CEA2EBF1BCC907 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:34,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:34,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:34,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FE9C40F6B0A14603B0CEA2EBF1BCC907
2016-01-14 18:31:34,766 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:34,766 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FE9C40F6B0A14603B0CEA2EBF1BCC907
2016-01-14 18:31:34,766 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:34,766 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796294571_0001_1_00_000000_0
2016-01-14 18:31:34,766 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:34,767 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@9d0db6e
2016-01-14 18:31:34,768 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@9d0db6e
2016-01-14 18:31:34,768 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:34,768 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:34,770 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:34,770 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: FE9C40F6B0A14603B0CEA2EBF1BCC907
2016-01-14 18:31:34,770 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C3DDDA4F73D64D2EB84AFF24A6D1B848
2016-01-14 18:31:34,770 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C3DDDA4F73D64D2EB84AFF24A6D1B848
2016-01-14 18:31:34,771 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@74e47803; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@438348e9; comparator=cascading.tuple.hadoop.util.TupleComparator@6027db9f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:34,771 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:34,771 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:34,771 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:34,784 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:34,784 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:34,784 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:34,784 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:34,784 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:34,785 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:34,785 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:34,785 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E263AFBD7D8741639EEA6F9E113F2556, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:34,790 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796294571_0001_1_00_000000_0
2016-01-14 18:31:34,790 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E263AFBD7D8741639EEA6F9E113F2556, mem on close (mb), free: 122, total: 372, max: 672
2016-01-14 18:31:34,790 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:34,790 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:34,791 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:34,791 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:34,792 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:34 Completed running task attempt: attempt_1452796294571_0001_1_00_000000_0
2016-01-14 18:31:34,794 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796294571_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:34,795 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796294785, Process_Duration=5, Process_End_Time=1452796294790, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:34,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796294571_0001_1_00, eventCount=1
2016-01-14 18:31:34,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E263AFBD7D8741639EEA6F9E113F2556, taskAttemptId=attempt_1452796294571_0001_1_00_000000_0, startTime=1452796294751, finishTime=1452796294795, timeTaken=44, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796294785, Process_Duration=5, Process_End_Time=1452796294790, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:34,795 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294571_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:34,796 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:34,796 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796294571_0001_1_00_000000_0
2016-01-14 18:31:34,796 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:34,796 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:TASK_FINISHED]: vertexName=E263AFBD7D8741639EEA6F9E113F2556, taskId=task_1452796294571_0001_1_00_000000, startTime=1452796294751, finishTime=1452796294796, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452796294571_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796294785, Process_Duration=5, Process_End_Time=1452796294790, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:34,796 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:34,796 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294571_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:34,797 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:34,797 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:34,797 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556] : 1
2016-01-14 18:31:34,797 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556]
2016-01-14 18:31:34,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:34,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:VERTEX_FINISHED]: vertexName=E263AFBD7D8741639EEA6F9E113F2556, vertexId=vertex_1452796294571_0001_1_00, initRequestedTime=1452796294733, initedTime=1452796294741, startRequestedTime=1452796294742, startedTime=1452796294742, finishTime=1452796294797, timeTaken=55, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796294785, Process_Duration=5, Process_End_Time=1452796294790, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796294751, firstTasksToStart=[ task_1452796294571_0001_1_00_000000 ], lastTaskFinishTime=1452796294795, lastTasksToFinish=[ task_1452796294571_0001_1_00_000000 ], minTaskDuration=44, maxTaskDuration=44, avgTaskDuration=44.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796294571_0001_1_00_000000 ], longestDurationTasks=[ task_1452796294571_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:34,798 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:34,798 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:34,798 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] attempt: attempt_1452796294571_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:34,798 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F4056EEC45C84599BC7AEDF9DFD1056B
2016-01-14 18:31:34,798 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B]
2016-01-14 18:31:34,798 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796294571_0001_00_000001
2016-01-14 18:31:34,799 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:34,798 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796294571_0001_00_000001, stoppedTime=1452796294798, exitStatus=0
2016-01-14 18:31:34,799 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:34,799 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796294571_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:34,799 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796294571_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:34,799 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796294571_0001_00_000001, Exiting
2016-01-14 18:31:34,799 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796294571_0001_00_000001
2016-01-14 18:31:34,800 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796294571_0001_00_000001
2016-01-14 18:31:34,799 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:34,800 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:34,800 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:TASK_STARTED]: vertexName=F4056EEC45C84599BC7AEDF9DFD1056B, taskId=task_1452796294571_0001_1_01_000000, scheduledTime=1452796294800, launchTime=1452796294800
2016-01-14 18:31:34,801 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294571_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:34,801 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796294571_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:34,801 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B]
2016-01-14 18:31:34,802 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796294571_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:34,802 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:34,802 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796294571_0001_00_000001 completed successfully
2016-01-14 18:31:34,802 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: F4056EEC45C84599BC7AEDF9DFD1056B, VertexParallelism: 1, TaskAttemptID:attempt_1452796294571_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E263AFBD7D8741639EEA6F9E113F2556, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1909503A65EB4B89B03FF53A9AE6A9E9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:34,803 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294571_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:34,803 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:34,803 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:34,803 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796294571_0001_00_000002
2016-01-14 18:31:34,804 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:34,805 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796294571_0001_00_000002
2016-01-14 18:31:34,806 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:34,807 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:34,808 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:34,808 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796294571_0001_00_000002, launchTime=1452796294807
2016-01-14 18:31:34,808 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796294571_0001_00_000002
2016-01-14 18:31:34,808 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796294571_0001_1_01_000000_0] to container: [container_1452796294571_0001_00_000002]
2016-01-14 18:31:34,808 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:34,809 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796294571_0001_1_01_000000_0] started. Is using containerId: [container_1452796294571_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:34,809 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F4056EEC45C84599BC7AEDF9DFD1056B, taskAttemptId=attempt_1452796294571_0001_1_01_000000_0, startTime=1452796294809, containerId=container_1452796294571_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796294571_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:34,809 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:34,809 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294571_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:34,809 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294571_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:34,809 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796294571_0001_00_000002 given task: attempt_1452796294571_0001_1_01_000000_0
2016-01-14 18:31:34,809 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796294571_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796294571_0001_1_01_000000_0
2016-01-14 18:31:34 Starting to run new task attempt: attempt_1452796294571_0001_1_01_000000_0
2016-01-14 18:31:34,810 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796294571_0001_1_01_000000_0
2016-01-14 18:31:34,810 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:34,810 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:34,810 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:34,810 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:34,810 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:34,810 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:34,810 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C] every/(1/1) ...mblyplatform/0_VALUES_ALL, VertexName: F4056EEC45C84599BC7AEDF9DFD1056B, VertexParallelism: 1, TaskAttemptID:attempt_1452796294571_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E263AFBD7D8741639EEA6F9E113F2556, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1909503A65EB4B89B03FF53A9AE6A9E9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:34,810 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:34,811 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796294571_0001_1_01_000000_0
2016-01-14 18:31:34,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:34,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,811 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796294571_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:34,811 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1909503A65EB4B89B03FF53A9AE6A9E9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:34,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:34,814 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:34,814 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1909503A65EB4B89B03FF53A9AE6A9E9
2016-01-14 18:31:34,814 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:34,814 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1909503A65EB4B89B03FF53A9AE6A9E9
2016-01-14 18:31:34,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E263AFBD7D8741639EEA6F9E113F2556, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:34,818 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:34,819 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E263AFBD7D8741639EEA6F9E113F2556
2016-01-14 18:31:34,821 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:34,821 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F4056EEC45C84599BC7AEDF9DFD1056B, ordinal: 1
2016-01-14 18:31:34,821 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4056EEC45C84599BC7AEDF9DFD1056B, mem on start (mb), free: 116, total: 372, max: 672
2016-01-14 18:31:34,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:34,821 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:34,821 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:34,822 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E263AFBD7D8741639EEA6F9E113F2556
2016-01-14 18:31:34,822 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:34,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E263AFBD7D8741639EEA6F9E113F2556
2016-01-14 18:31:34,822 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1909503A65EB4B89B03FF53A9AE6A9E9
2016-01-14 18:31:34,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:34,823 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:34,823 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:34,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:34,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:34,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:34,823 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:34,824 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:34,824 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1909503A65EB4B89B03FF53A9AE6A9E9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:34,824 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E263AFBD7D8741639EEA6F9E113F2556, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:34,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:34,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E263AFBD7D8741639EEA6F9E113F2556 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:34,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:34,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:34,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E263AFBD7D8741639EEA6F9E113F2556
2016-01-14 18:31:34,825 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:34,825 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:34,826 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E263AFBD7D8741639EEA6F9E113F2556 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:34,826 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:34,826 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:34,826 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:34,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E263AFBD7D8741639EEA6F9E113F2556
2016-01-14 18:31:34,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:34,827 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796294571_0001_1_01_000000_0
2016-01-14 18:31:34,830 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796294571_0001_1_00_000000_0_10189, runDuration: 31211, ]
2016-01-14 18:31:34,830 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:34,831 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C3DDDA4F73D64D2EB84AFF24A6D1B848
2016-01-14 18:31:34,831 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL"], id: 1909503A65EB4B89B03FF53A9AE6A9E9
2016-01-14 18:31:34,831 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_ALL"]
2016-01-14 18:31:34,831 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C3DDDA4F73D64D2EB84AFF24A6D1B848, for 1 inputs
2016-01-14 18:31:34,832 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796294571_0001_1_00_000000_0_10189] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796294832, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:34,832 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E263AFBD7D8741639EEA6F9E113F2556
2016-01-14 18:31:34,832 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:34,832 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E263AFBD7D8741639EEA6F9E113F2556] #32 in 1ms
2016-01-14 18:31:34,833 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E263AFBD7D8741639EEA6F9E113F2556] #32, status:false, isInterrupted:false
2016-01-14 18:31:34,833 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:34,833 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:34,833 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:34,834 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:34,834 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:34,834 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E263AFBD7D8741639EEA6F9E113F2556
2016-01-14 18:31:34,834 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:34,834 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F4056EEC45C84599BC7AEDF9DFD1056B, all 1 inputs ready in: 00:00:00.003
2016-01-14 18:31:34,835 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@74ce9321; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@261d9770
2016-01-14 18:31:34,835 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:34,835 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:34,836 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:34,836 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:34,836 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796294571_0001_1_01_000000_0
2016-01-14 18:31:34,836 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796294571_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:34,840 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796294571_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:34,841 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:34,841 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F4056EEC45C84599BC7AEDF9DFD1056B, taskAttemptId=attempt_1452796294571_0001_1_01_000000_0, startTime=1452796294809, finishTime=1452796294841, timeTaken=32, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796294834, Process_Duration=6, Process_End_Time=1452796294840, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:34,841 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294571_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:34,841 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796294571_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:34,841 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:34,841 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:TASK_FINISHED]: vertexName=F4056EEC45C84599BC7AEDF9DFD1056B, taskId=task_1452796294571_0001_1_01_000000, startTime=1452796294809, finishTime=1452796294841, timeTaken=32, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:34,842 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796294571_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:34,842 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:34,842 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] : 1
2016-01-14 18:31:34,842 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] because task failed: task_1452796294571_0001_1_01_000000
2016-01-14 18:31:34,842 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:34,842 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:34,842 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B]
2016-01-14 18:31:34,842 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=1909503A65EB4B89B03FF53A9AE6A9E9, vertexId=vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B]
2016-01-14 18:31:34,844 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:34,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:VERTEX_FINISHED]: vertexName=F4056EEC45C84599BC7AEDF9DFD1056B, vertexId=vertex_1452796294571_0001_1_01, initRequestedTime=1452796294734, initedTime=1452796294740, startRequestedTime=1452796294745, startedTime=1452796294745, finishTime=1452796294843, timeTaken=98, status=FAILED, diagnostics=Task failed, taskId=task_1452796294571_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796294809, firstTasksToStart=[ task_1452796294571_0001_1_01_000000 ], lastTaskFinishTime=1452796294841, lastTasksToFinish=[ task_1452796294571_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:34,844 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:34,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796294571_0001_00_000002, stoppedTime=1452796294844, exitStatus=0
2016-01-14 18:31:34,844 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:34,844 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796294571_0001_1_01_000000_0 failed: true container: container_1452796294571_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:34,845 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:34,845 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:34,845 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796294571_0001_1
2016-01-14 18:31:34,845 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 1909503A65EB4B89B03FF53A9AE6A9E9 for vertex: vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B]
2016-01-14 18:31:34,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796294834, Process_Duration=6, Process_End_Time=1452796294840, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:34,844 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:34,845 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F4056EEC45C84599BC7AEDF9DFD1056B, mem on close (mb), free: 109, total: 372, max: 672
2016-01-14 18:31:34,845 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796294571_0001_1_00 [E263AFBD7D8741639EEA6F9E113F2556]
2016-01-14 18:31:34,846 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=F4056EEC45C84599BC7AEDF9DFD1056B, index=1
2016-01-14 18:31:34,846 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E263AFBD7D8741639EEA6F9E113F2556
2016-01-14 18:31:34,847 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:34,847 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:34,847 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796294571_0001_1, queueSize=0
2016-01-14 18:31:34,847 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:34,848 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796294571_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796294571_0001_1, startTime=1452796294732, finishTime=1452796294846, timeTaken=114, status=FAILED, diagnostics=Vertex failed, vertexName=F4056EEC45C84599BC7AEDF9DFD1056B, vertexId=vertex_1452796294571_0001_1_01, diagnostics=[Task failed, taskId=task_1452796294571_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796294571_0001_1_01 [F4056EEC45C84599BC7AEDF9DFD1056B] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796294785, Process_Duration=5, Process_End_Time=1452796294790, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:34,848 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796294571_0001_1 finished with state: FAILED
2016-01-14 18:31:34,848 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796294571_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:34,848 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796294571_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:34 Completed Dag: dag_1452796294571_0001_1
2016-01-14 18:31:34,848 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796294571_0001_1, dagState=FAILED
2016-01-14 18:31:34,848 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796294571_0001_1_post
2016-01-14 18:31:34,849 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:34,849 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:34,849 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:34,848 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:34,849 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:34,847 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:34,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=F4056EEC45C84599BC7AEDF9DFD1056B, sourceVertex=E263AFBD7D8741639EEA6F9E113F2556
2016-01-14 18:31:34,850 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=F4056EEC45C84599BC7AEDF9DFD1056B, sourceVertex=1909503A65EB4B89B03FF53A9AE6A9E9
2016-01-14 18:31:34,851 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796294571_0001_00_000002
2016-01-14 18:31:34,851 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796294571_0001_00_000002
2016-01-14 18:31:34,851 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796294571_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:34,851 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796294571_0001_00_000002 completed but with errors
2016-01-14 18:31:34,852 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796294571_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:34,852 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796294571_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:34,852 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796294571_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:34,856 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...mblyplatform/0_VALUES_ALL, applicationId=application_1452796294571_0001
2016-01-14 18:31:34,858 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:34,858 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:34,858 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:34,858 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:34,858 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:34,859 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:34,859 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:34,859 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:34,859 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:34,860 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:34,860 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:34,860 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:34,861 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:34,866 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/D53DB5CE7041437F8CC5AE9D449B6F30/B92CFC98102244B6A0355D249AB6066C/.tez/application_1452796294571_0001
2016-01-14 18:31:34,866 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:34,866 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...mblyplatform/0_VALUES_ALL
2016-01-14 18:31:34,866 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:34,867 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.319
2016-01-14 18:31:34,867 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_ALL
2016-01-14 18:31:34,908 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:34,908 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:34,943 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 18:31:34,947 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 18:31:34,948 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:34,948 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:34,948 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:34,955 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:34,955 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:34,962 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:34,967 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE
2016-01-14 18:31:35,013 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:35,014 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:35,014 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE"]
2016-01-14 18:31:35,014 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:35,014 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:35,014 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:35,015 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/0_VALUES_REPLACE
2016-01-14 18:31:35,015 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:35,030 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:35,034 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:35,034 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:35,035 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:35,043 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE/.tez/application_1452796295034_0001 doesn't exist and is created
2016-01-14 18:31:35,057 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:35,058 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:35,058 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE/.tez/application_1452796295034_0001
2016-01-14 18:31:35,059 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796295034_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:35,075 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:35,076 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:35,077 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:35,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:35,078 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE/.tez/application_1452796295034_0001/localmode-log-dir
2016-01-14 18:31:35,078 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE/.tez/application_1452796295034_0001/localmode-log-dir/history.txt.appattempt_1452796295034_0001_000000, maxErrors=10
2016-01-14 18:31:35,078 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:35,079 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796295034_0001_000000, appSubmitTime=1452796295059, launchTime=1452796295059
2016-01-14 18:31:35,083 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:35,083 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:35,084 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:35,084 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:35,088 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:38510
2016-01-14 18:31:35,094 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:35,096 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796295034_0001_000000, startTime=1452796295096
2016-01-14 18:31:35,096 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:35,096 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:35,158 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:35,158 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:35,158 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1452796295034_0001, dagName=[001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE] every/(1/1) ...platform/0_VALUES_REPLACE
2016-01-14 18:31:35,174 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE] every/(1/1) ...platform/0_VALUES_REPLACE
2016-01-14 18:31:35,176 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796295034_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE/.tez/application_1452796295034_0001/localmode-log-dir/dag_1452796295034_0001_1.dot
2016-01-14 18:31:35,176 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE/.tez/application_1452796295034_0001/localmode-log-dir/dag_1452796295034_0001_1-tez-dag.pb.txt
2016-01-14 18:31:35,185 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796295034_0001_1
2016-01-14 18:31:35,185 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:35,185 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE] every/(1/1) ...platform/0_VALUES_REPLACE
2016-01-14 18:31:35 Running Dag: dag_1452796295034_0001_1
2016-01-14 18:31:35,194 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796295034_0001_1, submitTime=1452796295174
2016-01-14 18:31:35,195 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:35,195 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E7059AA846EE4754885A372B0ACA6309
2016-01-14 18:31:35,195 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:35,196 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796295034_0001_1, initTime=1452796295195
2016-01-14 18:31:35,196 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796295034_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:35,196 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:35,196 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:DAG_STARTED]: dagID=dag_1452796295034_0001_1, startTime=1452796295196
2016-01-14 18:31:35,196 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:35,196 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:35,197 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796295034_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:35,197 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C98D8D8704C04474863A31BFA30BEB79 : {6E3D475C0E474089A861754D5B3EA0E4={InputName=6E3D475C0E474089A861754D5B3EA0E4}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:35,197 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6E3D475C0E474089A861754D5B3EA0E4, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:35,197 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79]
2016-01-14 18:31:35,197 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79]
2016-01-14 18:31:35,197 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79]
2016-01-14 18:31:35,197 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79]. Starting root input initializers: 1
2016-01-14 18:31:35,197 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1452796295034_0001, dagName=[001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE] every/(1/1) ...platform/0_VALUES_REPLACE
2016-01-14 18:31:35,199 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:35,200 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309]
2016-01-14 18:31:35,201 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6E3D475C0E474089A861754D5B3EA0E4 on vertex vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79]
2016-01-14 18:31:35,203 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:35,203 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:35,203 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309]
2016-01-14 18:31:35,203 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309]
2016-01-14 18:31:35,203 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309]
2016-01-14 18:31:35,203 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=1B511EB5DCAA475886857DDD9B4A5616, vertexId=vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:35,204 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=1B511EB5DCAA475886857DDD9B4A5616, vertexId=vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309]
2016-01-14 18:31:35,204 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6E3D475C0E474089A861754D5B3EA0E4 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:35,204 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:35,205 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:35,206 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=1B511EB5DCAA475886857DDD9B4A5616, vertexName=E7059AA846EE4754885A372B0ACA6309, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:35,206 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=1B511EB5DCAA475886857DDD9B4A5616, vertexId=vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309]
2016-01-14 18:31:35,207 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:35,207 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:35,207 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E7059AA846EE4754885A372B0ACA6309, vertexId=vertex_1452796295034_0001_1_01, initRequestedTime=1452796295200, initedTime=1452796295207, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:35,207 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:35,207 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6E3D475C0E474089A861754D5B3EA0E4 on vertex vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79]
2016-01-14 18:31:35,208 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:35,208 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6E3D475C0E474089A861754D5B3EA0E4=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:35,208 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79] parallelism set to 1
2016-01-14 18:31:35,208 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:35,209 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796295034_0001_1_00, eventCount=1
2016-01-14 18:31:35,209 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:35,209 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C98D8D8704C04474863A31BFA30BEB79, vertexId=vertex_1452796295034_0001_1_00, initRequestedTime=1452796295197, initedTime=1452796295209, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:35,209 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79] with distanceFromRoot: 0
2016-01-14 18:31:35,209 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:35,210 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C98D8D8704C04474863A31BFA30BEB79
2016-01-14 18:31:35,210 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79]
2016-01-14 18:31:35,210 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:35,210 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796295034_0001_1_00, startRequestedTime=1452796295209, startedTime=1452796295209
2016-01-14 18:31:35,210 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:35,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:35,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:TASK_STARTED]: vertexName=C98D8D8704C04474863A31BFA30BEB79, taskId=task_1452796295034_0001_1_00_000000, scheduledTime=1452796295211, launchTime=1452796295211
2016-01-14 18:31:35,211 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295034_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:35,212 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796295034_0001_1_00 for vertex: vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] numStartedSources: 1 numSources: 1
2016-01-14 18:31:35,212 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] with distanceFromRoot: 1
2016-01-14 18:31:35,212 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79]
2016-01-14 18:31:35,212 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C98D8D8704C04474863A31BFA30BEB79 in vertex: E7059AA846EE4754885A372B0ACA6309
2016-01-14 18:31:35,213 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E7059AA846EE4754885A372B0ACA6309 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:35,213 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796295034_0001_1_01, startRequestedTime=1452796295212, startedTime=1452796295212
2016-01-14 18:31:35,213 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:35,213 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:35,214 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: C98D8D8704C04474863A31BFA30BEB79, VertexParallelism: 1, TaskAttemptID:attempt_1452796295034_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6E3D475C0E474089A861754D5B3EA0E4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E7059AA846EE4754885A372B0ACA6309, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:35,214 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295034_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:35,215 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:35,215 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:35,216 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:35,216 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:35,216 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796295034_0001_00_000001
2016-01-14 18:31:35,216 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:35,217 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796295034_0001, with dag id: dag_1452796295034_0001_1
2016-01-14 18:31:35,218 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796295034_0001_00_000001
2016-01-14 18:31:35,218 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:35,219 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:35,219 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:35,219 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796295034_0001_00_000001, launchTime=1452796295219
2016-01-14 18:31:35,220 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796295034_0001_00_000001
2016-01-14 18:31:35,221 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796295034_0001_1_00_000000_0] to container: [container_1452796295034_0001_00_000001]
2016-01-14 18:31:35,221 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:35,221 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796295034_0001_00_000001 given task: attempt_1452796295034_0001_1_00_000000_0
2016-01-14 18:31:35,221 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796295034_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796295034_0001_1_00_000000_0
2016-01-14 18:31:35 Starting to run new task attempt: attempt_1452796295034_0001_1_00_000000_0
2016-01-14 18:31:35,222 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796295034_0001_1_00_000000_0] started. Is using containerId: [container_1452796295034_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:35,222 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796295034_0001_1_00_000000_0
2016-01-14 18:31:35,223 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:35,223 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:35,223 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:35,223 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:35,224 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:35,224 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:35,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: C98D8D8704C04474863A31BFA30BEB79, VertexParallelism: 1, TaskAttemptID:attempt_1452796295034_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6E3D475C0E474089A861754D5B3EA0E4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E7059AA846EE4754885A372B0ACA6309, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:35,224 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:35,224 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:35,224 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796295034_0001_1_00_000000_0
2016-01-14 18:31:35,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:35,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,225 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C98D8D8704C04474863A31BFA30BEB79, taskAttemptId=attempt_1452796295034_0001_1_00_000000_0, startTime=1452796295222, containerId=container_1452796295034_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796295034_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:35,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6E3D475C0E474089A861754D5B3EA0E4, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:35,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:35,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6E3D475C0E474089A861754D5B3EA0E4
2016-01-14 18:31:35,225 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6E3D475C0E474089A861754D5B3EA0E4
2016-01-14 18:31:35,226 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796295034_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:35,226 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295034_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:35,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,226 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295034_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:35,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E7059AA846EE4754885A372B0ACA6309, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:35,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:35,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E7059AA846EE4754885A372B0ACA6309
2016-01-14 18:31:35,228 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:35,229 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:35,229 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E7059AA846EE4754885A372B0ACA6309
2016-01-14 18:31:35,230 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E7059AA846EE4754885A372B0ACA6309
2016-01-14 18:31:35,230 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:35,230 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6E3D475C0E474089A861754D5B3EA0E4
2016-01-14 18:31:35,230 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6E3D475C0E474089A861754D5B3EA0E4
2016-01-14 18:31:35,232 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C98D8D8704C04474863A31BFA30BEB79, ordinal: 0
2016-01-14 18:31:35,232 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C98D8D8704C04474863A31BFA30BEB79, mem on start (mb), free: 227, total: 372, max: 672
2016-01-14 18:31:35,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,232 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:35,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:35,233 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:35,233 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:35,233 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:35,233 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:35,233 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:35,233 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:35,233 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:35,233 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6E3D475C0E474089A861754D5B3EA0E4, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:35,233 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E7059AA846EE4754885A372B0ACA6309, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:35,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:35,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6E3D475C0E474089A861754D5B3EA0E4 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:35,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6E3D475C0E474089A861754D5B3EA0E4
2016-01-14 18:31:35,234 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:35,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6E3D475C0E474089A861754D5B3EA0E4
2016-01-14 18:31:35,234 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:35,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:35,235 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:35,235 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796295034_0001_1_00_000000_0
2016-01-14 18:31:35,235 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:35,236 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@14fe5f60
2016-01-14 18:31:35,237 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@14fe5f60
2016-01-14 18:31:35,237 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:35,238 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:35,239 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:35,241 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 6E3D475C0E474089A861754D5B3EA0E4
2016-01-14 18:31:35,241 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 7F532B9FB54E468CBE4D49EA9AD23EBE
2016-01-14 18:31:35,241 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 7F532B9FB54E468CBE4D49EA9AD23EBE
2016-01-14 18:31:35,242 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@64dd7a0e; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@87860de; comparator=cascading.tuple.hadoop.util.TupleComparator@171f932a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:35,242 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:35,242 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:35,242 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:35,255 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:35,255 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:35,255 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:35,255 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:35,255 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:35,256 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:35,256 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:35,256 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C98D8D8704C04474863A31BFA30BEB79, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:35,259 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796295034_0001_1_00_000000_0
2016-01-14 18:31:35,259 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C98D8D8704C04474863A31BFA30BEB79, mem on close (mb), free: 125, total: 372, max: 672
2016-01-14 18:31:35,259 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:35,259 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:35,259 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:35,259 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:35,261 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:35 Completed running task attempt: attempt_1452796295034_0001_1_00_000000_0
2016-01-14 18:31:35,264 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796295034_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:35,264 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796295256, Process_Duration=3, Process_End_Time=1452796295259, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:35,264 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796295034_0001_1_00, eventCount=1
2016-01-14 18:31:35,264 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C98D8D8704C04474863A31BFA30BEB79, taskAttemptId=attempt_1452796295034_0001_1_00_000000_0, startTime=1452796295222, finishTime=1452796295264, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796295256, Process_Duration=3, Process_End_Time=1452796295259, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:35,265 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295034_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:35,265 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:35,265 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:35,265 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796295034_0001_1_00_000000_0
2016-01-14 18:31:35,266 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:TASK_FINISHED]: vertexName=C98D8D8704C04474863A31BFA30BEB79, taskId=task_1452796295034_0001_1_00_000000, startTime=1452796295222, finishTime=1452796295265, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1452796295034_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796295256, Process_Duration=3, Process_End_Time=1452796295259, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:35,266 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295034_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:35,266 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79] : 1
2016-01-14 18:31:35,266 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79]
2016-01-14 18:31:35,266 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:VERTEX_FINISHED]: vertexName=C98D8D8704C04474863A31BFA30BEB79, vertexId=vertex_1452796295034_0001_1_00, initRequestedTime=1452796295197, initedTime=1452796295209, startRequestedTime=1452796295209, startedTime=1452796295209, finishTime=1452796295266, timeTaken=57, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796295256, Process_Duration=3, Process_End_Time=1452796295259, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796295222, firstTasksToStart=[ task_1452796295034_0001_1_00_000000 ], lastTaskFinishTime=1452796295264, lastTasksToFinish=[ task_1452796295034_0001_1_00_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796295034_0001_1_00_000000 ], longestDurationTasks=[ task_1452796295034_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:35,267 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:35,267 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:35,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:35,267 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:35,268 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:35,267 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:35,268 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796295034_0001_00_000001, stoppedTime=1452796295268, exitStatus=0
2016-01-14 18:31:35,268 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:35,268 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] attempt: attempt_1452796295034_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:35,268 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E7059AA846EE4754885A372B0ACA6309
2016-01-14 18:31:35,268 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309]
2016-01-14 18:31:35,269 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:35,269 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:35,269 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796295034_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:35,269 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:TASK_STARTED]: vertexName=E7059AA846EE4754885A372B0ACA6309, taskId=task_1452796295034_0001_1_01_000000, scheduledTime=1452796295269, launchTime=1452796295269
2016-01-14 18:31:35,269 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:35,269 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295034_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:35,269 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309]
2016-01-14 18:31:35,270 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:35,270 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: E7059AA846EE4754885A372B0ACA6309, VertexParallelism: 1, TaskAttemptID:attempt_1452796295034_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C98D8D8704C04474863A31BFA30BEB79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1B511EB5DCAA475886857DDD9B4A5616, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:35,270 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295034_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:35,270 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:35,271 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796295034_0001_00_000002
2016-01-14 18:31:35,271 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:35,272 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796295034_0001_00_000001
2016-01-14 18:31:35,272 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796295034_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:35,272 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796295034_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:35,272 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796295034_0001_00_000001, Exiting
2016-01-14 18:31:35,272 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796295034_0001_00_000001
2016-01-14 18:31:35,272 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796295034_0001_00_000001
2016-01-14 18:31:35,273 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796295034_0001_00_000002
2016-01-14 18:31:35,273 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796295034_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:35,273 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796295034_0001_00_000001 completed successfully
2016-01-14 18:31:35,274 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:35,274 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:35,277 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:35,277 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:35,277 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796295034_0001_00_000002, launchTime=1452796295277
2016-01-14 18:31:35,277 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796295034_0001_00_000002
2016-01-14 18:31:35,278 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796295034_0001_1_01_000000_0] to container: [container_1452796295034_0001_00_000002]
2016-01-14 18:31:35,278 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:35,278 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796295034_0001_00_000002 given task: attempt_1452796295034_0001_1_01_000000_0
2016-01-14 18:31:35,278 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796295034_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796295034_0001_1_01_000000_0
2016-01-14 18:31:35,278 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796295034_0001_1_01_000000_0] started. Is using containerId: [container_1452796295034_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:35,278 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E7059AA846EE4754885A372B0ACA6309, taskAttemptId=attempt_1452796295034_0001_1_01_000000_0, startTime=1452796295278, containerId=container_1452796295034_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796295034_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:35,278 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:35,279 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295034_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:35,279 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295034_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:35 Starting to run new task attempt: attempt_1452796295034_0001_1_01_000000_0
2016-01-14 18:31:35,279 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796295034_0001_1_01_000000_0
2016-01-14 18:31:35,279 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:35,279 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:35,279 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:35,280 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:35,280 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:35,280 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:35,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE] every/(1/1) ...platform/0_VALUES_REPLACE, VertexName: E7059AA846EE4754885A372B0ACA6309, VertexParallelism: 1, TaskAttemptID:attempt_1452796295034_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C98D8D8704C04474863A31BFA30BEB79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=1B511EB5DCAA475886857DDD9B4A5616, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:35,280 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:35,280 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796295034_0001_1_01_000000_0
2016-01-14 18:31:35,280 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:35,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,281 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796295034_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:35,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=1B511EB5DCAA475886857DDD9B4A5616, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:35,281 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C98D8D8704C04474863A31BFA30BEB79, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:35,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:35,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:35,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C98D8D8704C04474863A31BFA30BEB79
2016-01-14 18:31:35,282 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 1B511EB5DCAA475886857DDD9B4A5616
2016-01-14 18:31:35,283 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:35,283 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 1B511EB5DCAA475886857DDD9B4A5616
2016-01-14 18:31:35,284 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:35,284 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:35,285 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C98D8D8704C04474863A31BFA30BEB79
2016-01-14 18:31:35,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C98D8D8704C04474863A31BFA30BEB79
2016-01-14 18:31:35,286 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:35,290 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E7059AA846EE4754885A372B0ACA6309, ordinal: 1
2016-01-14 18:31:35,290 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E7059AA846EE4754885A372B0ACA6309, mem on start (mb), free: 117, total: 372, max: 672
2016-01-14 18:31:35,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:35,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:35,294 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:35,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 1B511EB5DCAA475886857DDD9B4A5616
2016-01-14 18:31:35,294 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:35,294 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:35,295 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:35,295 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:35,295 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:35,295 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:35,295 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:35,295 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 1B511EB5DCAA475886857DDD9B4A5616, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:35,295 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C98D8D8704C04474863A31BFA30BEB79, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:35,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:35,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C98D8D8704C04474863A31BFA30BEB79 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:35,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:35,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:35,295 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C98D8D8704C04474863A31BFA30BEB79
2016-01-14 18:31:35,296 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:35,296 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:35,297 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C98D8D8704C04474863A31BFA30BEB79 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:35,297 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:35,297 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:35,297 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:35,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C98D8D8704C04474863A31BFA30BEB79
2016-01-14 18:31:35,298 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:35,298 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796295034_0001_1_01_000000_0
2016-01-14 18:31:35,300 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796295034_0001_1_00_000000_0_10195, runDuration: 35377, ]
2016-01-14 18:31:35,302 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796295034_0001_1_00_000000_0_10195] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796295302, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:35,302 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C98D8D8704C04474863A31BFA30BEB79
2016-01-14 18:31:35,302 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:35,302 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:35,303 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C98D8D8704C04474863A31BFA30BEB79] #33 in 2ms
2016-01-14 18:31:35,303 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 7F532B9FB54E468CBE4D49EA9AD23EBE
2016-01-14 18:31:35,303 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE"], id: 1B511EB5DCAA475886857DDD9B4A5616
2016-01-14 18:31:35,303 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_REPLACE"]
2016-01-14 18:31:35,303 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 7F532B9FB54E468CBE4D49EA9AD23EBE, for 1 inputs
2016-01-14 18:31:35,303 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C98D8D8704C04474863A31BFA30BEB79] #33, status:false, isInterrupted:false
2016-01-14 18:31:35,304 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:35,304 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:35,304 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:35,304 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:35,304 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:35,304 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C98D8D8704C04474863A31BFA30BEB79
2016-01-14 18:31:35,304 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:35,304 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E7059AA846EE4754885A372B0ACA6309, all 1 inputs ready in: 00:00:00.001
2016-01-14 18:31:35,305 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1593a9e; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@9175bc3
2016-01-14 18:31:35,306 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:35,308 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:35,310 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:35,311 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:35,311 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796295034_0001_1_01_000000_0
2016-01-14 18:31:35,311 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796295034_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:35,312 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796295034_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:35,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796295304, Process_Duration=8, Process_End_Time=1452796295312, Read_Duration=1, Tuples_Read=1]]
2016-01-14 18:31:35,315 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E7059AA846EE4754885A372B0ACA6309, mem on close (mb), free: 110, total: 372, max: 672
2016-01-14 18:31:35,315 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E7059AA846EE4754885A372B0ACA6309, index=1
2016-01-14 18:31:35,315 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:35,315 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E7059AA846EE4754885A372B0ACA6309, taskAttemptId=attempt_1452796295034_0001_1_01_000000_0, startTime=1452796295278, finishTime=1452796295315, timeTaken=37, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796295304, Process_Duration=8, Process_End_Time=1452796295312, Read_Duration=1, Tuples_Read=1
2016-01-14 18:31:35,315 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295034_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:35,316 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796295034_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:35,315 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C98D8D8704C04474863A31BFA30BEB79
2016-01-14 18:31:35,316 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:35,316 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:TASK_FINISHED]: vertexName=E7059AA846EE4754885A372B0ACA6309, taskId=task_1452796295034_0001_1_01_000000, startTime=1452796295278, finishTime=1452796295316, timeTaken=38, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:35,316 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:35,316 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295034_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:35,317 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] : 1
2016-01-14 18:31:35,317 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] because task failed: task_1452796295034_0001_1_01_000000
2016-01-14 18:31:35,317 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:35,317 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:35,317 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:35,317 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309]
2016-01-14 18:31:35,317 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:35,317 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:35,317 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=1B511EB5DCAA475886857DDD9B4A5616, vertexId=vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309]
2016-01-14 18:31:35,317 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:35,318 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:35,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E7059AA846EE4754885A372B0ACA6309, sourceVertex=C98D8D8704C04474863A31BFA30BEB79
2016-01-14 18:31:35,318 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E7059AA846EE4754885A372B0ACA6309, sourceVertex=1B511EB5DCAA475886857DDD9B4A5616
2016-01-14 18:31:35,318 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796295034_0001_00_000002
2016-01-14 18:31:35,318 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796295034_0001_00_000002
2016-01-14 18:31:35,318 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796295034_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:35,319 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796295034_0001_00_000002 completed but with errors
2016-01-14 18:31:35,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:VERTEX_FINISHED]: vertexName=E7059AA846EE4754885A372B0ACA6309, vertexId=vertex_1452796295034_0001_1_01, initRequestedTime=1452796295200, initedTime=1452796295207, startRequestedTime=1452796295212, startedTime=1452796295212, finishTime=1452796295318, timeTaken=106, status=FAILED, diagnostics=Task failed, taskId=task_1452796295034_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796295278, firstTasksToStart=[ task_1452796295034_0001_1_01_000000 ], lastTaskFinishTime=1452796295314, lastTasksToFinish=[ task_1452796295034_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:35,319 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:35,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796295034_0001_00_000002, stoppedTime=1452796295319, exitStatus=0
2016-01-14 18:31:35,319 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:35,319 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796295034_0001_1_01_000000_0 failed: true container: container_1452796295034_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:35,319 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:35,320 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:35,320 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796295034_0001_1
2016-01-14 18:31:35,320 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 1B511EB5DCAA475886857DDD9B4A5616 for vertex: vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309]
2016-01-14 18:31:35,320 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796295034_0001_1_00 [C98D8D8704C04474863A31BFA30BEB79]
2016-01-14 18:31:35,321 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796295034_0001_1, queueSize=0
2016-01-14 18:31:35,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295034_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796295034_0001_1, startTime=1452796295196, finishTime=1452796295320, timeTaken=124, status=FAILED, diagnostics=Vertex failed, vertexName=E7059AA846EE4754885A372B0ACA6309, vertexId=vertex_1452796295034_0001_1_01, diagnostics=[Task failed, taskId=task_1452796295034_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796295034_0001_1_01 [E7059AA846EE4754885A372B0ACA6309] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390070272, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796295256, Process_Duration=3, Process_End_Time=1452796295259, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:35,322 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796295034_0001_1 finished with state: FAILED
2016-01-14 18:31:35,322 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796295034_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:35,322 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796295034_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:35,322 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295034_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:35,322 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796295034_0001_00_000002
2016-01-14 18:31:35 Completed Dag: dag_1452796295034_0001_1
2016-01-14 18:31:35,322 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796295034_0001_1, dagState=FAILED
2016-01-14 18:31:35,322 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796295034_0001_1_post
2016-01-14 18:31:35,322 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:35,322 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:35,323 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295034_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:35,319 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:35,323 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:35,323 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:35,331 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/0_VALUES_REPLACE, applicationId=application_1452796295034_0001
2016-01-14 18:31:35,332 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:35,333 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:35,333 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:35,333 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:35,333 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:35,333 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:35,333 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:35,334 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:35,334 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:35,334 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:35,334 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:35,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:35,335 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:35,340 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/001B66C7A48C47EEA0B9091C309C5DCD/E616939E241F4370815DDCF79A3730AE/.tez/application_1452796295034_0001
2016-01-14 18:31:35,340 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:35,342 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...platform/0_VALUES_REPLACE
2016-01-14 18:31:35,342 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:35,342 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.329
2016-01-14 18:31:35,343 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_REPLACE
2016-01-14 18:31:35,368 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:35,368 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:35,408 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 18:31:35,414 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 18:31:35,414 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:35,414 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:35,415 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:35,430 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:35,438 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP
2016-01-14 18:31:35,487 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:35,488 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:35,488 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP"]
2016-01-14 18:31:35,488 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:35,488 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:35,488 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:35,489 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 18:31:35,489 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:35,501 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:35,505 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:35,506 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:35,506 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:35,515 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D/.tez/application_1452796295506_0001 doesn't exist and is created
2016-01-14 18:31:35,529 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:35,529 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:35,529 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D/.tez/application_1452796295506_0001
2016-01-14 18:31:35,530 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796295506_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:35,540 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:35,540 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:35,541 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:35,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:35,542 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D/.tez/application_1452796295506_0001/localmode-log-dir
2016-01-14 18:31:35,542 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D/.tez/application_1452796295506_0001/localmode-log-dir/history.txt.appattempt_1452796295506_0001_000000, maxErrors=10
2016-01-14 18:31:35,542 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:35,543 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796295506_0001_000000, appSubmitTime=1452796295530, launchTime=1452796295530
2016-01-14 18:31:35,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:35,545 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:35,546 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:35,548 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59328
2016-01-14 18:31:35,549 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:35,551 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:35,552 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796295506_0001_000000, startTime=1452796295552
2016-01-14 18:31:35,552 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:35,552 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:35,572 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:35,572 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:35,629 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:35,629 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:35,629 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1452796295506_0001, dagName=[19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 18:31:35,644 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 18:31:35,645 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796295506_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D/.tez/application_1452796295506_0001/localmode-log-dir/dag_1452796295506_0001_1.dot
2016-01-14 18:31:35,646 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D/.tez/application_1452796295506_0001/localmode-log-dir/dag_1452796295506_0001_1-tez-dag.pb.txt
2016-01-14 18:31:35,652 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796295506_0001_1
2016-01-14 18:31:35,652 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:35,653 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 18:31:35 Running Dag: dag_1452796295506_0001_1
2016-01-14 18:31:35,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796295506_0001_1, submitTime=1452796295644
2016-01-14 18:31:35,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:35,662 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B932900E35A442BBABAF83DD8500B023
2016-01-14 18:31:35,662 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:35,663 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:35,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796295506_0001_1, initTime=1452796295662
2016-01-14 18:31:35,663 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796295506_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:35,663 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:DAG_STARTED]: dagID=dag_1452796295506_0001_1, startTime=1452796295663
2016-01-14 18:31:35,663 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:35,663 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:35,664 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796295506_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:35,664 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F7D168847AD14C52851384B0BE6E2059 : {FB25C6CEB7184152BC588639540D2AD5={InputName=FB25C6CEB7184152BC588639540D2AD5}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:35,664 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: FB25C6CEB7184152BC588639540D2AD5, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:35,664 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059]
2016-01-14 18:31:35,664 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059]
2016-01-14 18:31:35,664 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059]
2016-01-14 18:31:35,664 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059]. Starting root input initializers: 1
2016-01-14 18:31:35,665 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1452796295506_0001, dagName=[19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D] every/(1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 18:31:35,665 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:35,665 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023]
2016-01-14 18:31:35,666 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: FB25C6CEB7184152BC588639540D2AD5 on vertex vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059]
2016-01-14 18:31:35,668 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:35,668 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:35,668 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023]
2016-01-14 18:31:35,668 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023]
2016-01-14 18:31:35,669 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023]
2016-01-14 18:31:35,669 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=4A124A7E0B9B4349BC9F9BCF30184275, vertexId=vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:35,669 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=4A124A7E0B9B4349BC9F9BCF30184275, vertexId=vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023]
2016-01-14 18:31:35,669 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input FB25C6CEB7184152BC588639540D2AD5 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:35,670 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:35,670 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:35,671 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=4A124A7E0B9B4349BC9F9BCF30184275, vertexName=B932900E35A442BBABAF83DD8500B023, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:35,671 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=4A124A7E0B9B4349BC9F9BCF30184275, vertexId=vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023]
2016-01-14 18:31:35,671 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:35,672 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B932900E35A442BBABAF83DD8500B023, vertexId=vertex_1452796295506_0001_1_01, initRequestedTime=1452796295665, initedTime=1452796295671, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:35,672 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:35,673 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:35,673 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: FB25C6CEB7184152BC588639540D2AD5 on vertex vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059]
2016-01-14 18:31:35,673 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:35,673 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {FB25C6CEB7184152BC588639540D2AD5=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:35,674 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059] parallelism set to 1
2016-01-14 18:31:35,674 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:35,674 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796295506_0001_1_00, eventCount=1
2016-01-14 18:31:35,674 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:35,675 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F7D168847AD14C52851384B0BE6E2059, vertexId=vertex_1452796295506_0001_1_00, initRequestedTime=1452796295664, initedTime=1452796295674, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:35,675 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059] with distanceFromRoot: 0
2016-01-14 18:31:35,675 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:35,675 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F7D168847AD14C52851384B0BE6E2059
2016-01-14 18:31:35,675 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059]
2016-01-14 18:31:35,676 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:35,676 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796295506_0001_1_00, startRequestedTime=1452796295675, startedTime=1452796295675
2016-01-14 18:31:35,676 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:35,676 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:35,677 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:TASK_STARTED]: vertexName=F7D168847AD14C52851384B0BE6E2059, taskId=task_1452796295506_0001_1_00_000000, scheduledTime=1452796295676, launchTime=1452796295676
2016-01-14 18:31:35,677 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295506_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:35,677 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796295506_0001_1_00 for vertex: vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] numStartedSources: 1 numSources: 1
2016-01-14 18:31:35,677 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] with distanceFromRoot: 1
2016-01-14 18:31:35,677 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059]
2016-01-14 18:31:35,678 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F7D168847AD14C52851384B0BE6E2059 in vertex: B932900E35A442BBABAF83DD8500B023
2016-01-14 18:31:35,678 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B932900E35A442BBABAF83DD8500B023 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:35,678 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:35,678 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796295506_0001_1_01, startRequestedTime=1452796295678, startedTime=1452796295678
2016-01-14 18:31:35,679 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:35,679 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: F7D168847AD14C52851384B0BE6E2059, VertexParallelism: 1, TaskAttemptID:attempt_1452796295506_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FB25C6CEB7184152BC588639540D2AD5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B932900E35A442BBABAF83DD8500B023, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:35,679 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295506_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:35,679 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:35,680 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:35,680 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:35,680 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:35,681 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796295506_0001_00_000001
2016-01-14 18:31:35,681 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:35,682 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796295506_0001_00_000001
2016-01-14 18:31:35,682 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:35,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:35,683 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:35,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796295506_0001_00_000001, launchTime=1452796295683
2016-01-14 18:31:35,684 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796295506_0001_00_000001
2016-01-14 18:31:35,684 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796295506_0001_1_00_000000_0] to container: [container_1452796295506_0001_00_000001]
2016-01-14 18:31:35,684 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796295506_0001, with dag id: dag_1452796295506_0001_1
2016-01-14 18:31:35,684 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:35,684 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796295506_0001_00_000001 given task: attempt_1452796295506_0001_1_00_000000_0
2016-01-14 18:31:35,685 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796295506_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796295506_0001_1_00_000000_0
2016-01-14 18:31:35,685 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796295506_0001_1_00_000000_0] started. Is using containerId: [container_1452796295506_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:35,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:35,685 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F7D168847AD14C52851384B0BE6E2059, taskAttemptId=attempt_1452796295506_0001_1_00_000000_0, startTime=1452796295685, containerId=container_1452796295506_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796295506_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:35,685 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295506_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:35,686 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295506_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:35 Starting to run new task attempt: attempt_1452796295506_0001_1_00_000000_0
2016-01-14 18:31:35,686 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796295506_0001_1_00_000000_0
2016-01-14 18:31:35,686 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:35,686 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:35,686 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:35,686 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:35,686 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:35,687 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:35,687 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: F7D168847AD14C52851384B0BE6E2059, VertexParallelism: 1, TaskAttemptID:attempt_1452796295506_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FB25C6CEB7184152BC588639540D2AD5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B932900E35A442BBABAF83DD8500B023, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:35,687 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:35,688 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796295506_0001_1_00_000000_0
2016-01-14 18:31:35,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:35,688 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FB25C6CEB7184152BC588639540D2AD5, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:35,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:35,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FB25C6CEB7184152BC588639540D2AD5
2016-01-14 18:31:35,689 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796295506_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:35,690 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: FB25C6CEB7184152BC588639540D2AD5
2016-01-14 18:31:35,691 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B932900E35A442BBABAF83DD8500B023, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:35,692 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:35,692 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: FB25C6CEB7184152BC588639540D2AD5
2016-01-14 18:31:35,692 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:35,692 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FB25C6CEB7184152BC588639540D2AD5
2016-01-14 18:31:35,695 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F7D168847AD14C52851384B0BE6E2059, ordinal: 0
2016-01-14 18:31:35,695 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7D168847AD14C52851384B0BE6E2059, mem on start (mb), free: 229, total: 372, max: 672
2016-01-14 18:31:35,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:35,695 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:35,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:35,696 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B932900E35A442BBABAF83DD8500B023
2016-01-14 18:31:35,697 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:35,698 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B932900E35A442BBABAF83DD8500B023
2016-01-14 18:31:35,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B932900E35A442BBABAF83DD8500B023
2016-01-14 18:31:35,698 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:35,698 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:35,698 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:35,698 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:35,698 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:35,698 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:35,699 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:35,699 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FB25C6CEB7184152BC588639540D2AD5, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:35,699 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B932900E35A442BBABAF83DD8500B023, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:35,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:35,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FB25C6CEB7184152BC588639540D2AD5 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:35,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:35,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:35,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FB25C6CEB7184152BC588639540D2AD5
2016-01-14 18:31:35,699 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:35,699 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FB25C6CEB7184152BC588639540D2AD5
2016-01-14 18:31:35,700 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:35,700 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796295506_0001_1_00_000000_0
2016-01-14 18:31:35,700 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:35,700 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@18429cd5
2016-01-14 18:31:35,702 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@18429cd5
2016-01-14 18:31:35,702 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:35,702 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:35,703 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:35,704 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: FB25C6CEB7184152BC588639540D2AD5
2016-01-14 18:31:35,704 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 07E9A37EA434483EA3F3A6DF5488EF86
2016-01-14 18:31:35,704 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 07E9A37EA434483EA3F3A6DF5488EF86
2016-01-14 18:31:35,705 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@18a5e38e; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@32fe8c46; comparator=cascading.tuple.hadoop.util.TupleComparator@65ab61d1; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:35,705 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:35,705 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:35,705 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:35,718 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:35,718 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:35,718 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:35,718 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:35,718 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:35,719 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:35,719 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:35,719 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F7D168847AD14C52851384B0BE6E2059, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:35,722 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796295506_0001_1_00_000000_0
2016-01-14 18:31:35,722 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F7D168847AD14C52851384B0BE6E2059, mem on close (mb), free: 126, total: 372, max: 672
2016-01-14 18:31:35,723 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:35,723 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:35,723 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:35,723 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:35,725 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:35 Completed running task attempt: attempt_1452796295506_0001_1_00_000000_0
2016-01-14 18:31:35,727 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796295506_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:35,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796295506_0001_1_00, eventCount=1
2016-01-14 18:31:35,728 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:35,728 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F7D168847AD14C52851384B0BE6E2059, taskAttemptId=attempt_1452796295506_0001_1_00_000000_0, startTime=1452796295685, finishTime=1452796295728, timeTaken=43, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796295720, Process_Duration=2, Process_End_Time=1452796295722, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:35,728 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295506_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:35,728 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796295506_0001_1_00_000000_0
2016-01-14 18:31:35,728 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:35,729 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:35,729 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:TASK_FINISHED]: vertexName=F7D168847AD14C52851384B0BE6E2059, taskId=task_1452796295506_0001_1_00_000000, startTime=1452796295685, finishTime=1452796295728, timeTaken=43, status=SUCCEEDED, successfulAttemptID=attempt_1452796295506_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796295720, Process_Duration=2, Process_End_Time=1452796295722, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:35,729 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295506_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:35,729 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059] : 1
2016-01-14 18:31:35,729 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059]
2016-01-14 18:31:35,730 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:VERTEX_FINISHED]: vertexName=F7D168847AD14C52851384B0BE6E2059, vertexId=vertex_1452796295506_0001_1_00, initRequestedTime=1452796295664, initedTime=1452796295674, startRequestedTime=1452796295675, startedTime=1452796295675, finishTime=1452796295729, timeTaken=54, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796295720, Process_Duration=2, Process_End_Time=1452796295722, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796295685, firstTasksToStart=[ task_1452796295506_0001_1_00_000000 ], lastTaskFinishTime=1452796295728, lastTasksToFinish=[ task_1452796295506_0001_1_00_000000 ], minTaskDuration=43, maxTaskDuration=43, avgTaskDuration=43.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796295506_0001_1_00_000000 ], longestDurationTasks=[ task_1452796295506_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:35,730 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:35,730 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:35,731 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:35,731 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] attempt: attempt_1452796295506_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:35,731 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B932900E35A442BBABAF83DD8500B023
2016-01-14 18:31:35,731 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023]
2016-01-14 18:31:35,731 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796295720, Process_Duration=2, Process_End_Time=1452796295722, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:35,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:35,732 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:35,732 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:35,732 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796295506_0001_00_000001, stoppedTime=1452796295732, exitStatus=0
2016-01-14 18:31:35,732 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:35,733 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:35,733 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796295506_0001_00_000001
2016-01-14 18:31:35,733 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796295506_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:35,733 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796295506_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:35,733 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:TASK_STARTED]: vertexName=B932900E35A442BBABAF83DD8500B023, taskId=task_1452796295506_0001_1_01_000000, scheduledTime=1452796295733, launchTime=1452796295733
2016-01-14 18:31:35,733 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:35,734 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295506_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:35,734 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796295506_0001_00_000001, Exiting
2016-01-14 18:31:35,734 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796295506_0001_00_000001
2016-01-14 18:31:35,734 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796295506_0001_00_000001
2016-01-14 18:31:35,735 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023]
2016-01-14 18:31:35,735 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796295506_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:35,735 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: B932900E35A442BBABAF83DD8500B023, VertexParallelism: 1, TaskAttemptID:attempt_1452796295506_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7D168847AD14C52851384B0BE6E2059, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4A124A7E0B9B4349BC9F9BCF30184275, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:35,735 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295506_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:35,735 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:35,735 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796295506_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:35,736 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:35,736 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796295506_0001_00_000001 completed successfully
2016-01-14 18:31:35,736 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796295506_0001_00_000002
2016-01-14 18:31:35,736 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:35,737 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:35,738 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796295506_0001_00_000002
2016-01-14 18:31:35,738 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:35,740 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:35,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:35,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796295506_0001_00_000002, launchTime=1452796295740
2016-01-14 18:31:35,741 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796295506_0001_00_000002
2016-01-14 18:31:35,741 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796295506_0001_1_01_000000_0] to container: [container_1452796295506_0001_00_000002]
2016-01-14 18:31:35,742 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:35,742 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796295506_0001_1_01_000000_0] started. Is using containerId: [container_1452796295506_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:35,742 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B932900E35A442BBABAF83DD8500B023, taskAttemptId=attempt_1452796295506_0001_1_01_000000_0, startTime=1452796295742, containerId=container_1452796295506_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796295506_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:35,742 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:35,742 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796295506_0001_00_000002 given task: attempt_1452796295506_0001_1_01_000000_0
2016-01-14 18:31:35,742 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295506_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:35,743 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295506_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:35,743 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796295506_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796295506_0001_1_01_000000_0
2016-01-14 18:31:35 Starting to run new task attempt: attempt_1452796295506_0001_1_01_000000_0
2016-01-14 18:31:35,743 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796295506_0001_1_01_000000_0
2016-01-14 18:31:35,743 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:35,743 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:35,743 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:35,744 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:35,744 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:35,744 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:35,744 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D] every/(1/1) ...blyplatform/0_VALUES_SWAP, VertexName: B932900E35A442BBABAF83DD8500B023, VertexParallelism: 1, TaskAttemptID:attempt_1452796295506_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F7D168847AD14C52851384B0BE6E2059, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=4A124A7E0B9B4349BC9F9BCF30184275, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:35,744 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:35,745 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796295506_0001_1_01_000000_0
2016-01-14 18:31:35,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:35,745 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,746 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796295506_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:35,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=4A124A7E0B9B4349BC9F9BCF30184275, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:35,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,746 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F7D168847AD14C52851384B0BE6E2059, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:35,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:35,747 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F7D168847AD14C52851384B0BE6E2059
2016-01-14 18:31:35,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:35,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 4A124A7E0B9B4349BC9F9BCF30184275
2016-01-14 18:31:35,748 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:35,749 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 4A124A7E0B9B4349BC9F9BCF30184275
2016-01-14 18:31:35,749 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:35,749 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:35,750 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:35,750 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F7D168847AD14C52851384B0BE6E2059
2016-01-14 18:31:35,750 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F7D168847AD14C52851384B0BE6E2059
2016-01-14 18:31:35,755 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B932900E35A442BBABAF83DD8500B023, ordinal: 1
2016-01-14 18:31:35,755 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B932900E35A442BBABAF83DD8500B023, mem on start (mb), free: 119, total: 372, max: 672
2016-01-14 18:31:35,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:35,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:35,755 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:35,758 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:35,758 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 4A124A7E0B9B4349BC9F9BCF30184275
2016-01-14 18:31:35,759 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:35,759 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:35,759 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:35,759 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:35,759 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:35,759 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:35,760 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:35,760 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 4A124A7E0B9B4349BC9F9BCF30184275, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:35,760 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F7D168847AD14C52851384B0BE6E2059, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:35,760 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:35,760 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F7D168847AD14C52851384B0BE6E2059
2016-01-14 18:31:35,760 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F7D168847AD14C52851384B0BE6E2059 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:35,761 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:35,761 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:35,762 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F7D168847AD14C52851384B0BE6E2059 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:35,762 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:35,762 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:35,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:35,762 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:35,762 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:35,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F7D168847AD14C52851384B0BE6E2059
2016-01-14 18:31:35,763 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:35,764 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796295506_0001_1_01_000000_0
2016-01-14 18:31:35,768 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:35,768 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 07E9A37EA434483EA3F3A6DF5488EF86
2016-01-14 18:31:35,769 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP"], id: 4A124A7E0B9B4349BC9F9BCF30184275
2016-01-14 18:31:35,769 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_VALUES_SWAP"]
2016-01-14 18:31:35,769 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 07E9A37EA434483EA3F3A6DF5488EF86, for 1 inputs
2016-01-14 18:31:35,769 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796295506_0001_1_00_000000_0_10201, runDuration: 30008, ]
2016-01-14 18:31:35,771 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796295506_0001_1_00_000000_0_10201] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796295771, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:35,771 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F7D168847AD14C52851384B0BE6E2059
2016-01-14 18:31:35,771 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:35,771 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F7D168847AD14C52851384B0BE6E2059] #34 in 1ms
2016-01-14 18:31:35,771 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F7D168847AD14C52851384B0BE6E2059] #34, status:false, isInterrupted:false
2016-01-14 18:31:35,772 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:35,772 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:35,772 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:35,772 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:35,773 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:35,773 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F7D168847AD14C52851384B0BE6E2059
2016-01-14 18:31:35,773 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:35,773 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B932900E35A442BBABAF83DD8500B023, all 1 inputs ready in: 00:00:00.004
2016-01-14 18:31:35,774 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@24beec84; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7299e0f4
2016-01-14 18:31:35,775 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:35,778 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:35,781 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:35,782 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:35,782 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796295506_0001_1_01_000000_0
2016-01-14 18:31:35,782 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796295506_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:35,783 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796295506_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:35,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796295773, Process_Duration=10, Process_End_Time=1452796295783, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:35,786 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B932900E35A442BBABAF83DD8500B023, mem on close (mb), free: 112, total: 372, max: 672
2016-01-14 18:31:35,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B932900E35A442BBABAF83DD8500B023, taskAttemptId=attempt_1452796295506_0001_1_01_000000_0, startTime=1452796295742, finishTime=1452796295786, timeTaken=44, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796295773, Process_Duration=10, Process_End_Time=1452796295783, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:35,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:35,787 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295506_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:35,787 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796295506_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:35,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:TASK_FINISHED]: vertexName=B932900E35A442BBABAF83DD8500B023, taskId=task_1452796295506_0001_1_01_000000, startTime=1452796295742, finishTime=1452796295787, timeTaken=45, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:35,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:35,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B932900E35A442BBABAF83DD8500B023, index=1
2016-01-14 18:31:35,787 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295506_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:35,788 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:35,788 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] : 1
2016-01-14 18:31:35,789 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] because task failed: task_1452796295506_0001_1_01_000000
2016-01-14 18:31:35,789 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:35,788 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F7D168847AD14C52851384B0BE6E2059
2016-01-14 18:31:35,789 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:35,789 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023]
2016-01-14 18:31:35,789 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:35,789 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=4A124A7E0B9B4349BC9F9BCF30184275, vertexId=vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023]
2016-01-14 18:31:35,791 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:VERTEX_FINISHED]: vertexName=B932900E35A442BBABAF83DD8500B023, vertexId=vertex_1452796295506_0001_1_01, initRequestedTime=1452796295665, initedTime=1452796295671, startRequestedTime=1452796295678, startedTime=1452796295678, finishTime=1452796295790, timeTaken=112, status=FAILED, diagnostics=Task failed, taskId=task_1452796295506_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796295742, firstTasksToStart=[ task_1452796295506_0001_1_01_000000 ], lastTaskFinishTime=1452796295786, lastTasksToFinish=[ task_1452796295506_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:35,791 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:35,791 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:35,791 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:35,791 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:35,792 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:35,792 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:35,792 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:35,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B932900E35A442BBABAF83DD8500B023, sourceVertex=F7D168847AD14C52851384B0BE6E2059
2016-01-14 18:31:35,792 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796295506_0001_00_000002, stoppedTime=1452796295792, exitStatus=0
2016-01-14 18:31:35,793 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:35,793 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796295506_0001_1_01_000000_0 failed: true container: container_1452796295506_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:35,792 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B932900E35A442BBABAF83DD8500B023, sourceVertex=4A124A7E0B9B4349BC9F9BCF30184275
2016-01-14 18:31:35,793 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:35,793 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:35,793 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796295506_0001_1
2016-01-14 18:31:35,793 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796295506_0001_1_00 [F7D168847AD14C52851384B0BE6E2059]
2016-01-14 18:31:35,793 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 4A124A7E0B9B4349BC9F9BCF30184275 for vertex: vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023]
2016-01-14 18:31:35,793 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796295506_0001_00_000002
2016-01-14 18:31:35,794 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796295506_0001_00_000002
2016-01-14 18:31:35,794 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796295506_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:35,794 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796295506_0001_00_000002 completed but with errors
2016-01-14 18:31:35,797 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796295506_0001_1, queueSize=0
2016-01-14 18:31:35,797 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:35,797 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295506_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796295506_0001_1, startTime=1452796295663, finishTime=1452796295794, timeTaken=131, status=FAILED, diagnostics=Vertex failed, vertexName=B932900E35A442BBABAF83DD8500B023, vertexId=vertex_1452796295506_0001_1_01, diagnostics=[Task failed, taskId=task_1452796295506_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796295506_0001_1_01 [B932900E35A442BBABAF83DD8500B023] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=390594560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796295720, Process_Duration=2, Process_End_Time=1452796295722, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:35,798 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796295506_0001_1 finished with state: FAILED
2016-01-14 18:31:35,798 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796295506_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:35,798 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796295506_0001_00_000002
2016-01-14 18:31:35,799 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796295506_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:35,800 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295506_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:35 Completed Dag: dag_1452796295506_0001_1
2016-01-14 18:31:35,802 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796295506_0001_1, dagState=FAILED
2016-01-14 18:31:35,802 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796295506_0001_1_post
2016-01-14 18:31:35,803 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:35,803 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:35,803 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295506_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:35,809 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/0_VALUES_SWAP, applicationId=application_1452796295506_0001
2016-01-14 18:31:35,810 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:35,810 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:35,811 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:35,811 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:35,811 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:35,811 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:35,812 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:35,812 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:35,812 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:35,812 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:35,813 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:35,813 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:35,813 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:35,819 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19EBB09D151D462C96979625CA70070B/FB2EADB9101149CD9B421D9E9A1D6D3D/.tez/application_1452796295506_0001
2016-01-14 18:31:35,819 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:35,820 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...blyplatform/0_VALUES_SWAP
2016-01-14 18:31:35,820 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:35,820 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.333
2016-01-14 18:31:35,821 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_VALUES_SWAP
2016-01-14 18:31:35,847 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:35,847 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:35,895 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.016
2016-01-14 18:31:35,898 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.018
2016-01-14 18:31:35,898 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:35,899 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:35,899 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:35,918 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:35,923 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1
2016-01-14 18:31:35,966 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:35,966 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:35,966 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1"]
2016-01-14 18:31:35,967 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:35,967 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:35,967 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:35,968 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...ssemblyplatform/0_ARGS_-1
2016-01-14 18:31:35,969 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:35,984 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:35,989 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:35,989 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:35,989 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:35,998 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037/.tez/application_1452796295989_0001 doesn't exist and is created
2016-01-14 18:31:36,012 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:36,012 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:36,013 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037/.tez/application_1452796295989_0001
2016-01-14 18:31:36,014 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796295989_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:36,024 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:36,024 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:36,025 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:36,025 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:36,026 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037/.tez/application_1452796295989_0001/localmode-log-dir
2016-01-14 18:31:36,026 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037/.tez/application_1452796295989_0001/localmode-log-dir/history.txt.appattempt_1452796295989_0001_000000, maxErrors=10
2016-01-14 18:31:36,027 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:36,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796295989_0001_000000, appSubmitTime=1452796296014, launchTime=1452796296014
2016-01-14 18:31:36,030 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:36931
2016-01-14 18:31:36,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:36,035 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:36,036 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:36,036 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:36,038 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:36,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796295989_0001_000000, startTime=1452796296039
2016-01-14 18:31:36,039 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:36,039 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:36,113 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:36,113 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:36,113 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1452796295989_0001, dagName=[C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-01-14 18:31:36,116 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:36,116 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:36,129 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-01-14 18:31:36,130 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796295989_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037/.tez/application_1452796295989_0001/localmode-log-dir/dag_1452796295989_0001_1.dot
2016-01-14 18:31:36,131 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037/.tez/application_1452796295989_0001/localmode-log-dir/dag_1452796295989_0001_1-tez-dag.pb.txt
2016-01-14 18:31:36,137 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796295989_0001_1
2016-01-14 18:31:36,137 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:36,137 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-01-14 18:31:36 Running Dag: dag_1452796295989_0001_1
2016-01-14 18:31:36,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:36,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796295989_0001_1, submitTime=1452796296129
2016-01-14 18:31:36,147 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex DCF319BC8F4144CABF289A64ACD8BDC7
2016-01-14 18:31:36,147 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:36,148 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:36,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796295989_0001_1, initTime=1452796296147
2016-01-14 18:31:36,148 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796295989_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:36,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:DAG_STARTED]: dagID=dag_1452796295989_0001_1, startTime=1452796296148
2016-01-14 18:31:36,148 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:36,148 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796295989_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:36,149 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E74EAD4343DF4834B3C0F84125B5BED8 : {E53FBBC735EF44AEAC1E26C18D17DDF1={InputName=E53FBBC735EF44AEAC1E26C18D17DDF1}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:36,149 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E53FBBC735EF44AEAC1E26C18D17DDF1, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:36,149 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8]
2016-01-14 18:31:36,149 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8]
2016-01-14 18:31:36,149 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8]
2016-01-14 18:31:36,149 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8]. Starting root input initializers: 1
2016-01-14 18:31:36,149 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:36,150 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:36,150 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7]
2016-01-14 18:31:36,150 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1452796295989_0001, dagName=[C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037] every/(1/1) ...ssemblyplatform/0_ARGS_-1
2016-01-14 18:31:36,153 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:36,153 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7]
2016-01-14 18:31:36,153 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7]
2016-01-14 18:31:36,153 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7]
2016-01-14 18:31:36,154 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=0320D37AD8A0474D8735318249F2102A, vertexId=vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:36,154 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=0320D37AD8A0474D8735318249F2102A, vertexId=vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7]
2016-01-14 18:31:36,155 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E53FBBC735EF44AEAC1E26C18D17DDF1 on vertex vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8]
2016-01-14 18:31:36,156 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=0320D37AD8A0474D8735318249F2102A, vertexName=DCF319BC8F4144CABF289A64ACD8BDC7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:36,156 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=0320D37AD8A0474D8735318249F2102A, vertexId=vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7]
2016-01-14 18:31:36,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:36,157 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:VERTEX_INITIALIZED]: vertexName=DCF319BC8F4144CABF289A64ACD8BDC7, vertexId=vertex_1452796295989_0001_1_01, initRequestedTime=1452796296150, initedTime=1452796296156, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:36,157 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:36,157 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:36,158 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E53FBBC735EF44AEAC1E26C18D17DDF1 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:36,160 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:36,161 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:36,162 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:36,162 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E53FBBC735EF44AEAC1E26C18D17DDF1 on vertex vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8]
2016-01-14 18:31:36,162 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:36,162 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E53FBBC735EF44AEAC1E26C18D17DDF1=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:36,162 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8] parallelism set to 1
2016-01-14 18:31:36,163 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:36,166 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796295989_0001_1_00, eventCount=1
2016-01-14 18:31:36,167 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E74EAD4343DF4834B3C0F84125B5BED8, vertexId=vertex_1452796295989_0001_1_00, initRequestedTime=1452796296149, initedTime=1452796296166, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:36,168 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8] with distanceFromRoot: 0
2016-01-14 18:31:36,167 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:36,168 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:36,168 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E74EAD4343DF4834B3C0F84125B5BED8
2016-01-14 18:31:36,169 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8]
2016-01-14 18:31:36,169 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796295989_0001_1_00, startRequestedTime=1452796296168, startedTime=1452796296168
2016-01-14 18:31:36,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:36,170 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:36,170 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796295989_0001, with dag id: dag_1452796295989_0001_1
2016-01-14 18:31:36,170 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:TASK_STARTED]: vertexName=E74EAD4343DF4834B3C0F84125B5BED8, taskId=task_1452796295989_0001_1_00_000000, scheduledTime=1452796296170, launchTime=1452796296170
2016-01-14 18:31:36,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:36,171 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295989_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:36,171 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796295989_0001_1_00 for vertex: vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7] numStartedSources: 1 numSources: 1
2016-01-14 18:31:36,171 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7] with distanceFromRoot: 1
2016-01-14 18:31:36,171 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8]
2016-01-14 18:31:36,172 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E74EAD4343DF4834B3C0F84125B5BED8 in vertex: DCF319BC8F4144CABF289A64ACD8BDC7
2016-01-14 18:31:36,173 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: DCF319BC8F4144CABF289A64ACD8BDC7 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:36,173 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796295989_0001_1_01, startRequestedTime=1452796296172, startedTime=1452796296172
2016-01-14 18:31:36,173 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:36,173 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:36,174 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: E74EAD4343DF4834B3C0F84125B5BED8, VertexParallelism: 1, TaskAttemptID:attempt_1452796295989_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E53FBBC735EF44AEAC1E26C18D17DDF1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DCF319BC8F4144CABF289A64ACD8BDC7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:36,174 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295989_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:36,174 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:36,175 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:36,176 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:36,176 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:36,176 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796295989_0001_00_000001
2016-01-14 18:31:36,176 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:36,178 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796295989_0001_00_000001
2016-01-14 18:31:36,178 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:36,179 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:36,179 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:36,179 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796295989_0001_00_000001, launchTime=1452796296179
2016-01-14 18:31:36,180 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796295989_0001_00_000001
2016-01-14 18:31:36,180 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796295989_0001_1_00_000000_0] to container: [container_1452796295989_0001_00_000001]
2016-01-14 18:31:36,180 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:36,180 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796295989_0001_00_000001 given task: attempt_1452796295989_0001_1_00_000000_0
2016-01-14 18:31:36,180 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796295989_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796295989_0001_1_00_000000_0
2016-01-14 18:31:36,181 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796295989_0001_1_00_000000_0] started. Is using containerId: [container_1452796295989_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:36 Starting to run new task attempt: attempt_1452796295989_0001_1_00_000000_0
2016-01-14 18:31:36,181 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E74EAD4343DF4834B3C0F84125B5BED8, taskAttemptId=attempt_1452796295989_0001_1_00_000000_0, startTime=1452796296180, containerId=container_1452796295989_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796295989_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:36,181 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796295989_0001_1_00_000000_0
2016-01-14 18:31:36,181 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:36,181 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:36,181 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295989_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:36,181 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:36,181 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295989_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:36,181 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:36,182 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:36,182 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:36,182 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:36,182 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: E74EAD4343DF4834B3C0F84125B5BED8, VertexParallelism: 1, TaskAttemptID:attempt_1452796295989_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E53FBBC735EF44AEAC1E26C18D17DDF1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=DCF319BC8F4144CABF289A64ACD8BDC7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:36,182 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:36,182 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796295989_0001_1_00_000000_0
2016-01-14 18:31:36,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:36,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:36,183 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796295989_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:36,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E53FBBC735EF44AEAC1E26C18D17DDF1, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:36,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=DCF319BC8F4144CABF289A64ACD8BDC7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:36,183 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:36,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:36,184 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E53FBBC735EF44AEAC1E26C18D17DDF1
2016-01-14 18:31:36,184 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E53FBBC735EF44AEAC1E26C18D17DDF1
2016-01-14 18:31:36,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:36,185 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: DCF319BC8F4144CABF289A64ACD8BDC7
2016-01-14 18:31:36,186 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:36,186 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E53FBBC735EF44AEAC1E26C18D17DDF1
2016-01-14 18:31:36,186 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E53FBBC735EF44AEAC1E26C18D17DDF1
2016-01-14 18:31:36,187 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:36,190 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:36,191 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: DCF319BC8F4144CABF289A64ACD8BDC7
2016-01-14 18:31:36,191 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: DCF319BC8F4144CABF289A64ACD8BDC7
2016-01-14 18:31:36,192 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E74EAD4343DF4834B3C0F84125B5BED8, ordinal: 0
2016-01-14 18:31:36,193 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E74EAD4343DF4834B3C0F84125B5BED8, mem on start (mb), free: 233, total: 374, max: 672
2016-01-14 18:31:36,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:36,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:36,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:36,193 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:36,193 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:36,193 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:36,194 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:36,194 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:36,194 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:36,194 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:36,194 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E53FBBC735EF44AEAC1E26C18D17DDF1, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:36,194 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, DCF319BC8F4144CABF289A64ACD8BDC7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:36,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:36,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E53FBBC735EF44AEAC1E26C18D17DDF1 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:36,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E53FBBC735EF44AEAC1E26C18D17DDF1
2016-01-14 18:31:36,195 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:36,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E53FBBC735EF44AEAC1E26C18D17DDF1
2016-01-14 18:31:36,195 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:36,196 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:36,196 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:36,196 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796295989_0001_1_00_000000_0
2016-01-14 18:31:36,196 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:36,197 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@323da969
2016-01-14 18:31:36,198 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@323da969
2016-01-14 18:31:36,198 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:36,199 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:36,201 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:36,203 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E53FBBC735EF44AEAC1E26C18D17DDF1
2016-01-14 18:31:36,203 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 0E643CEA35A74F8BA04858CDD71BA44F
2016-01-14 18:31:36,204 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 0E643CEA35A74F8BA04858CDD71BA44F
2016-01-14 18:31:36,204 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@5d226aa1; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7e869fcb; comparator=cascading.tuple.hadoop.util.TupleComparator@38650952; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:36,205 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:36,205 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:36,205 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:36,221 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:36,221 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:36,221 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:36,221 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:36,221 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:36,222 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:36,222 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:36,223 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E74EAD4343DF4834B3C0F84125B5BED8, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:36,226 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796295989_0001_1_00_000000_0
2016-01-14 18:31:36,226 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E74EAD4343DF4834B3C0F84125B5BED8, mem on close (mb), free: 130, total: 374, max: 672
2016-01-14 18:31:36,226 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:36,226 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:36,227 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:36,227 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:36,229 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:36 Completed running task attempt: attempt_1452796295989_0001_1_00_000000_0
2016-01-14 18:31:36,233 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796295989_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:36,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796295989_0001_1_00, eventCount=1
2016-01-14 18:31:36,234 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E74EAD4343DF4834B3C0F84125B5BED8, taskAttemptId=attempt_1452796295989_0001_1_00_000000_0, startTime=1452796296180, finishTime=1452796296234, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796296223, Process_Duration=3, Process_End_Time=1452796296226, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:36,234 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295989_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:36,234 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:36,234 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796295989_0001_1_00_000000_0
2016-01-14 18:31:36,235 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:TASK_FINISHED]: vertexName=E74EAD4343DF4834B3C0F84125B5BED8, taskId=task_1452796295989_0001_1_00_000000, startTime=1452796296180, finishTime=1452796296234, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452796295989_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796296223, Process_Duration=3, Process_End_Time=1452796296226, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:36,235 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295989_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:36,235 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8] : 1
2016-01-14 18:31:36,235 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8]
2016-01-14 18:31:36,236 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:VERTEX_FINISHED]: vertexName=E74EAD4343DF4834B3C0F84125B5BED8, vertexId=vertex_1452796295989_0001_1_00, initRequestedTime=1452796296149, initedTime=1452796296166, startRequestedTime=1452796296168, startedTime=1452796296168, finishTime=1452796296235, timeTaken=67, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796296223, Process_Duration=3, Process_End_Time=1452796296226, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796296180, firstTasksToStart=[ task_1452796295989_0001_1_00_000000 ], lastTaskFinishTime=1452796296234, lastTasksToFinish=[ task_1452796295989_0001_1_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796295989_0001_1_00_000000 ], longestDurationTasks=[ task_1452796295989_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:36,236 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:36,236 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:36,237 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:36,237 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:36,238 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:36,238 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7] attempt: attempt_1452796295989_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:36,238 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: DCF319BC8F4144CABF289A64ACD8BDC7
2016-01-14 18:31:36,238 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7]
2016-01-14 18:31:36,238 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:36,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796295989_0001_00_000001, stoppedTime=1452796296238, exitStatus=0
2016-01-14 18:31:36,239 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:36,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:TASK_STARTED]: vertexName=DCF319BC8F4144CABF289A64ACD8BDC7, taskId=task_1452796295989_0001_1_01_000000, scheduledTime=1452796296239, launchTime=1452796296239
2016-01-14 18:31:36,239 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295989_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:36,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:36,239 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:36,239 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796295989_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:36,239 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7]
2016-01-14 18:31:36,240 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:36,240 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: DCF319BC8F4144CABF289A64ACD8BDC7, VertexParallelism: 1, TaskAttemptID:attempt_1452796295989_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E74EAD4343DF4834B3C0F84125B5BED8, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0320D37AD8A0474D8735318249F2102A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:36,240 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295989_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:36,240 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:36,240 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796296223, Process_Duration=3, Process_End_Time=1452796296226, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:36,240 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:36,240 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:36,240 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796295989_0001_00_000002
2016-01-14 18:31:36,241 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:36,241 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796295989_0001_00_000001
2016-01-14 18:31:36,242 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796295989_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:36,242 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796295989_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:36,242 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796295989_0001_00_000002
2016-01-14 18:31:36,242 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796295989_0001_00_000001, Exiting
2016-01-14 18:31:36,242 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796295989_0001_00_000001
2016-01-14 18:31:36,243 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796295989_0001_00_000001
2016-01-14 18:31:36,242 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:36,244 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:36,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:36,245 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796295989_0001_00_000002, launchTime=1452796296244
2016-01-14 18:31:36,245 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796295989_0001_00_000002
2016-01-14 18:31:36,245 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796295989_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:36,246 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796295989_0001_00_000001 completed successfully
2016-01-14 18:31:36,246 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796295989_0001_1_01_000000_0] to container: [container_1452796295989_0001_00_000002]
2016-01-14 18:31:36,246 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:36,246 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796295989_0001_1_01_000000_0] started. Is using containerId: [container_1452796295989_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:36,246 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=DCF319BC8F4144CABF289A64ACD8BDC7, taskAttemptId=attempt_1452796295989_0001_1_01_000000_0, startTime=1452796296246, containerId=container_1452796295989_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796295989_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:36,247 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295989_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:36,247 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:36,247 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295989_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:36,247 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:36,247 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796295989_0001_00_000002 given task: attempt_1452796295989_0001_1_01_000000_0
2016-01-14 18:31:36,247 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796295989_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796295989_0001_1_01_000000_0
2016-01-14 18:31:36 Starting to run new task attempt: attempt_1452796295989_0001_1_01_000000_0
2016-01-14 18:31:36,247 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796295989_0001_1_01_000000_0
2016-01-14 18:31:36,247 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:36,248 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:36,248 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:36,248 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:36,248 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:36,248 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:36,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037] every/(1/1) ...ssemblyplatform/0_ARGS_-1, VertexName: DCF319BC8F4144CABF289A64ACD8BDC7, VertexParallelism: 1, TaskAttemptID:attempt_1452796295989_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E74EAD4343DF4834B3C0F84125B5BED8, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=0320D37AD8A0474D8735318249F2102A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:36,248 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:36,248 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796295989_0001_1_01_000000_0
2016-01-14 18:31:36,248 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:36,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:36,249 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796295989_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:36,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:36,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E74EAD4343DF4834B3C0F84125B5BED8, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:36,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:36,249 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E74EAD4343DF4834B3C0F84125B5BED8
2016-01-14 18:31:36,251 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:36,251 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:36,251 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E74EAD4343DF4834B3C0F84125B5BED8
2016-01-14 18:31:36,251 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E74EAD4343DF4834B3C0F84125B5BED8
2016-01-14 18:31:36,252 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:36,252 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=0320D37AD8A0474D8735318249F2102A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:36,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:36,253 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 0320D37AD8A0474D8735318249F2102A
2016-01-14 18:31:36,253 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:36,253 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 0320D37AD8A0474D8735318249F2102A
2016-01-14 18:31:36,255 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: DCF319BC8F4144CABF289A64ACD8BDC7, ordinal: 1
2016-01-14 18:31:36,255 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DCF319BC8F4144CABF289A64ACD8BDC7, mem on start (mb), free: 123, total: 374, max: 672
2016-01-14 18:31:36,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:36,255 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:36,256 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:36,262 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:36,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 0320D37AD8A0474D8735318249F2102A
2016-01-14 18:31:36,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:36,262 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:36,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:36,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:36,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:36,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:36,262 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:36,262 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E74EAD4343DF4834B3C0F84125B5BED8, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:36,262 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 0320D37AD8A0474D8735318249F2102A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:36,262 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:36,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E74EAD4343DF4834B3C0F84125B5BED8 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:36,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:36,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:36,263 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E74EAD4343DF4834B3C0F84125B5BED8
2016-01-14 18:31:36,263 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:36,263 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:36,264 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E74EAD4343DF4834B3C0F84125B5BED8 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:36,264 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:36,264 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:36,264 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:36,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E74EAD4343DF4834B3C0F84125B5BED8
2016-01-14 18:31:36,267 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:36,267 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796295989_0001_1_01_000000_0
2016-01-14 18:31:36,271 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:36,271 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 0E643CEA35A74F8BA04858CDD71BA44F
2016-01-14 18:31:36,271 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1"], id: 0320D37AD8A0474D8735318249F2102A
2016-01-14 18:31:36,271 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796295989_0001_1_00_000000_0_10207, runDuration: 46373, ]
2016-01-14 18:31:36,271 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_-1"]
2016-01-14 18:31:36,272 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 0E643CEA35A74F8BA04858CDD71BA44F, for 1 inputs
2016-01-14 18:31:36,273 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796295989_0001_1_00_000000_0_10207] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796296273, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:36,273 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E74EAD4343DF4834B3C0F84125B5BED8
2016-01-14 18:31:36,273 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:36,273 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E74EAD4343DF4834B3C0F84125B5BED8] #35 in 1ms
2016-01-14 18:31:36,273 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E74EAD4343DF4834B3C0F84125B5BED8] #35, status:false, isInterrupted:false
2016-01-14 18:31:36,274 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:36,275 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:36,275 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:36,275 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:36,275 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:36,275 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: DCF319BC8F4144CABF289A64ACD8BDC7, all 1 inputs ready in: 00:00:00.003
2016-01-14 18:31:36,276 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E74EAD4343DF4834B3C0F84125B5BED8
2016-01-14 18:31:36,276 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7f8f96f7; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2c00738e
2016-01-14 18:31:36,276 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:36,282 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:36,282 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:36,283 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796295989_0001_1_01_000000_0
2016-01-14 18:31:36,283 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796295989_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:36,284 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796295989_0001_1_01_000000_0
2016-01-14 18:31:36,284 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: DCF319BC8F4144CABF289A64ACD8BDC7, mem on close (mb), free: 116, total: 374, max: 672
2016-01-14 18:31:36,284 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E74EAD4343DF4834B3C0F84125B5BED8
2016-01-14 18:31:36,284 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:36,284 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:36,284 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:36,285 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:36,285 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:36 Completed running task attempt: attempt_1452796295989_0001_1_01_000000_0
2016-01-14 18:31:36,286 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796295989_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:36,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796296275, Process_Duration=9, Process_End_Time=1452796296284, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:36,286 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=DCF319BC8F4144CABF289A64ACD8BDC7, taskAttemptId=attempt_1452796295989_0001_1_01_000000_0, startTime=1452796296246, finishTime=1452796296286, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796296275, Process_Duration=9, Process_End_Time=1452796296284, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:36,287 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:36,287 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:36,286 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:36,287 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796295989_0001_00_000002
2016-01-14 18:31:36,287 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796295989_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:36,287 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:36,287 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796295989_0001_1_01_000000_0
2016-01-14 18:31:36,287 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:36,288 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:TASK_FINISHED]: vertexName=DCF319BC8F4144CABF289A64ACD8BDC7, taskId=task_1452796295989_0001_1_01_000000, startTime=1452796296246, finishTime=1452796296287, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1452796295989_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796296275, Process_Duration=9, Process_End_Time=1452796296284, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:36,288 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796295989_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:36,288 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7] : 1
2016-01-14 18:31:36,288 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7]
2016-01-14 18:31:36,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:36,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:VERTEX_FINISHED]: vertexName=DCF319BC8F4144CABF289A64ACD8BDC7, vertexId=vertex_1452796295989_0001_1_01, initRequestedTime=1452796296150, initedTime=1452796296156, startRequestedTime=1452796296172, startedTime=1452796296172, finishTime=1452796296288, timeTaken=116, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796296275, Process_Duration=9, Process_End_Time=1452796296284, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796296246, firstTasksToStart=[ task_1452796295989_0001_1_01_000000 ], lastTaskFinishTime=1452796296286, lastTasksToFinish=[ task_1452796295989_0001_1_01_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796295989_0001_1_01_000000 ], longestDurationTasks=[ task_1452796295989_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:36,289 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:36,289 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:36,289 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796295989_0001_00_000002, stoppedTime=1452796296289, exitStatus=0
2016-01-14 18:31:36,289 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:36,289 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:36,289 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:36,290 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796295989_0001_1
2016-01-14 18:31:36,290 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796295989_0001_1
2016-01-14 18:31:36,290 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 0320D37AD8A0474D8735318249F2102A for vertex: vertex_1452796295989_0001_1_01 [DCF319BC8F4144CABF289A64ACD8BDC7]
2016-01-14 18:31:36,296 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796295989_0001_1_00 [E74EAD4343DF4834B3C0F84125B5BED8]
2016-01-14 18:31:36,297 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796295989_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:36,297 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796295989_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:36,297 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796295989_0001_00_000002, Exiting
2016-01-14 18:31:36,298 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796295989_0001_00_000002
2016-01-14 18:31:36,298 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796295989_0001_00_000002
2016-01-14 18:31:36,298 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796295989_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:36,298 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796295989_0001_1, queueSize=0
2016-01-14 18:31:36,298 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796295989_0001_00_000002 completed successfully
2016-01-14 18:31:36,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:36,298 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796295989_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796295989_0001_1, startTime=1452796296148, finishTime=1452796296297, timeTaken=149, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=784334848, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=10, MERGE_PHASE_TIME=12, FIRST_EVENT_RECEIVED=9, LAST_EVENT_RECEIVED=9, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592592498, Process_Duration=12, Process_End_Time=2905592592510, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:36,298 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796295989_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:36,299 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796295989_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:36,299 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796295989_0001_00_000002
2016-01-14 18:31:36,299 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796295989_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:36 Completed Dag: dag_1452796295989_0001_1
2016-01-14 18:31:36,300 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796295989_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:36,300 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796295989_0001_1_post
2016-01-14 18:31:36,300 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:36,300 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:36,305 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...ssemblyplatform/0_ARGS_-1, applicationId=application_1452796295989_0001
2016-01-14 18:31:36,306 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:36,306 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:36,306 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:36,306 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:36,306 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:36,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:36,307 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:36,307 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:36,309 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:36,309 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:36,309 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:36,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:36,310 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:36,314 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/C127CE72451340C6BF87323E5C8B09F1/B2E33AD504B3441DA685FE01E8306037/.tez/application_1452796295989_0001
2016-01-14 18:31:36,315 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.349
2016-01-14 18:31:36,396 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_-1
2016-01-14 18:31:36,419 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:36,419 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:36,444 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 18:31:36,446 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-01-14 18:31:36,446 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:36,446 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:36,446 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_'field'
2016-01-14 18:31:36,468 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:36,468 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:36,498 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 18:31:36,506 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 18:31:36,506 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:36,506 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:36,507 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:36,521 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:36,528 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS
2016-01-14 18:31:36,575 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:36,575 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:36,576 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS"]
2016-01-14 18:31:36,576 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:36,576 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:36,576 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:36,577 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/0_ARGS_RESULTS
2016-01-14 18:31:36,577 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:36,590 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:36,594 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:36,594 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:36,594 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:36,603 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7/.tez/application_1452796296594_0001 doesn't exist and is created
2016-01-14 18:31:36,617 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:36,617 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:36,617 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7/.tez/application_1452796296594_0001
2016-01-14 18:31:36,619 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796296594_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:36,629 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:36,630 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:36,631 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:36,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:36,631 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7/.tez/application_1452796296594_0001/localmode-log-dir
2016-01-14 18:31:36,632 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7/.tez/application_1452796296594_0001/localmode-log-dir/history.txt.appattempt_1452796296594_0001_000000, maxErrors=10
2016-01-14 18:31:36,632 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:36,632 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796296594_0001_000000, appSubmitTime=1452796296618, launchTime=1452796296619
2016-01-14 18:31:36,636 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:36,638 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:36,640 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:50281
2016-01-14 18:31:36,642 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:36,642 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:36,644 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:36,645 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796296594_0001_000000, startTime=1452796296645
2016-01-14 18:31:36,645 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:36,645 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:36,710 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:36,710 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:36,717 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:36,717 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:36,717 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1452796296594_0001, dagName=[AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-01-14 18:31:36,732 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-01-14 18:31:36,734 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796296594_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7/.tez/application_1452796296594_0001/localmode-log-dir/dag_1452796296594_0001_1.dot
2016-01-14 18:31:36,734 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7/.tez/application_1452796296594_0001/localmode-log-dir/dag_1452796296594_0001_1-tez-dag.pb.txt
2016-01-14 18:31:36,740 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796296594_0001_1
2016-01-14 18:31:36,740 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:36,741 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-01-14 18:31:36 Running Dag: dag_1452796296594_0001_1
2016-01-14 18:31:36,750 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796296594_0001_1, submitTime=1452796296733
2016-01-14 18:31:36,750 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:36,751 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B1DC95D45A344354B912388F9E7CD937
2016-01-14 18:31:36,751 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:36,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796296594_0001_1, initTime=1452796296750
2016-01-14 18:31:36,751 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796296594_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:36,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:36,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:DAG_STARTED]: dagID=dag_1452796296594_0001_1, startTime=1452796296752
2016-01-14 18:31:36,752 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:36,752 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796296594_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:36,752 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B33BCEB1D6644C8791D37BCB18E94580 : {655E94FD9E1A4221B51F226B8845E112={InputName=655E94FD9E1A4221B51F226B8845E112}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:36,752 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 655E94FD9E1A4221B51F226B8845E112, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:36,752 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580]
2016-01-14 18:31:36,752 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580]
2016-01-14 18:31:36,752 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580]
2016-01-14 18:31:36,753 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580]. Starting root input initializers: 1
2016-01-14 18:31:36,753 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:36,753 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937]
2016-01-14 18:31:36,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:36,753 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 655E94FD9E1A4221B51F226B8845E112 on vertex vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580]
2016-01-14 18:31:36,755 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1452796296594_0001, dagName=[AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7] every/(1/1) ...lyplatform/0_ARGS_RESULTS
2016-01-14 18:31:36,756 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:36,756 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 655E94FD9E1A4221B51F226B8845E112 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:36,756 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:36,756 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937]
2016-01-14 18:31:36,756 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:36,757 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937]
2016-01-14 18:31:36,757 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937]
2016-01-14 18:31:36,757 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=CB081849CC3A491AA50F22707018164F, vertexId=vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:36,757 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=CB081849CC3A491AA50F22707018164F, vertexId=vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937]
2016-01-14 18:31:36,759 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=CB081849CC3A491AA50F22707018164F, vertexName=B1DC95D45A344354B912388F9E7CD937, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:36,759 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=CB081849CC3A491AA50F22707018164F, vertexId=vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937]
2016-01-14 18:31:36,757 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:36,760 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:36,760 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B1DC95D45A344354B912388F9E7CD937, vertexId=vertex_1452796296594_0001_1_01, initRequestedTime=1452796296753, initedTime=1452796296760, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:36,760 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:36,762 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:36,762 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 655E94FD9E1A4221B51F226B8845E112 on vertex vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580]
2016-01-14 18:31:36,762 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:36,762 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {655E94FD9E1A4221B51F226B8845E112=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:36,763 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580] parallelism set to 1
2016-01-14 18:31:36,763 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:36,763 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796296594_0001_1_00, eventCount=1
2016-01-14 18:31:36,763 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:36,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B33BCEB1D6644C8791D37BCB18E94580, vertexId=vertex_1452796296594_0001_1_00, initRequestedTime=1452796296752, initedTime=1452796296763, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:36,764 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580] with distanceFromRoot: 0
2016-01-14 18:31:36,764 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:36,764 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B33BCEB1D6644C8791D37BCB18E94580
2016-01-14 18:31:36,764 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580]
2016-01-14 18:31:36,764 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:36,765 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796296594_0001_1_00, startRequestedTime=1452796296764, startedTime=1452796296764
2016-01-14 18:31:36,765 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:36,765 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:36,765 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:TASK_STARTED]: vertexName=B33BCEB1D6644C8791D37BCB18E94580, taskId=task_1452796296594_0001_1_00_000000, scheduledTime=1452796296765, launchTime=1452796296765
2016-01-14 18:31:36,766 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796296594_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:36,766 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796296594_0001_1_00 for vertex: vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937] numStartedSources: 1 numSources: 1
2016-01-14 18:31:36,766 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937] with distanceFromRoot: 1
2016-01-14 18:31:36,766 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580]
2016-01-14 18:31:36,766 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B33BCEB1D6644C8791D37BCB18E94580 in vertex: B1DC95D45A344354B912388F9E7CD937
2016-01-14 18:31:36,767 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B1DC95D45A344354B912388F9E7CD937 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:36,767 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:36,767 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796296594_0001_1_01, startRequestedTime=1452796296766, startedTime=1452796296766
2016-01-14 18:31:36,767 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:36,768 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: B33BCEB1D6644C8791D37BCB18E94580, VertexParallelism: 1, TaskAttemptID:attempt_1452796296594_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=655E94FD9E1A4221B51F226B8845E112, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B1DC95D45A344354B912388F9E7CD937, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:36,768 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796296594_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:36,768 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:36,768 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:36,769 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:36,769 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:36,769 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796296594_0001_00_000001
2016-01-14 18:31:36,769 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:36,771 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796296594_0001_00_000001
2016-01-14 18:31:36,771 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:36,772 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:36,772 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796296594_0001_00_000001, launchTime=1452796296772
2016-01-14 18:31:36,772 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:36,773 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796296594_0001_00_000001
2016-01-14 18:31:36,773 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796296594_0001_1_00_000000_0] to container: [container_1452796296594_0001_00_000001]
2016-01-14 18:31:36,773 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:36,774 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796296594_0001_1_00_000000_0] started. Is using containerId: [container_1452796296594_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:36,774 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:36,774 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B33BCEB1D6644C8791D37BCB18E94580, taskAttemptId=attempt_1452796296594_0001_1_00_000000_0, startTime=1452796296773, containerId=container_1452796296594_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796296594_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:36,774 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796296594_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:36,774 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796296594_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:36,774 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796296594_0001_00_000001 given task: attempt_1452796296594_0001_1_00_000000_0
2016-01-14 18:31:36,774 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796296594_0001, with dag id: dag_1452796296594_0001_1
2016-01-14 18:31:36,774 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796296594_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796296594_0001_1_00_000000_0
2016-01-14 18:31:36 Starting to run new task attempt: attempt_1452796296594_0001_1_00_000000_0
2016-01-14 18:31:36,775 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796296594_0001_1_00_000000_0
2016-01-14 18:31:36,775 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:36,775 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:36,775 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:36,775 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:36,775 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:36,776 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:36,776 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: B33BCEB1D6644C8791D37BCB18E94580, VertexParallelism: 1, TaskAttemptID:attempt_1452796296594_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=655E94FD9E1A4221B51F226B8845E112, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B1DC95D45A344354B912388F9E7CD937, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:36,776 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:36,776 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796296594_0001_1_00_000000_0
2016-01-14 18:31:36,776 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:36,776 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:36,777 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796296594_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:36,777 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=655E94FD9E1A4221B51F226B8845E112, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:36,778 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:36,778 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 655E94FD9E1A4221B51F226B8845E112
2016-01-14 18:31:36,778 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 655E94FD9E1A4221B51F226B8845E112
2016-01-14 18:31:36,779 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:36,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B1DC95D45A344354B912388F9E7CD937, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:36,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:36,780 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B1DC95D45A344354B912388F9E7CD937
2016-01-14 18:31:36,781 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:36,781 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 655E94FD9E1A4221B51F226B8845E112
2016-01-14 18:31:36,782 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:36,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 655E94FD9E1A4221B51F226B8845E112
2016-01-14 18:31:36,782 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:36,782 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B1DC95D45A344354B912388F9E7CD937
2016-01-14 18:31:36,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B1DC95D45A344354B912388F9E7CD937
2016-01-14 18:31:36,784 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B33BCEB1D6644C8791D37BCB18E94580, ordinal: 0
2016-01-14 18:31:36,784 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B33BCEB1D6644C8791D37BCB18E94580, mem on start (mb), free: 196, total: 374, max: 672
2016-01-14 18:31:36,784 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:36,785 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:36,785 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:36,785 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:36,785 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:36,785 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:36,785 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:36,785 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:36,785 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:36,786 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:36,786 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 655E94FD9E1A4221B51F226B8845E112, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:36,786 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B1DC95D45A344354B912388F9E7CD937, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:36,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:36,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 655E94FD9E1A4221B51F226B8845E112 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:36,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 655E94FD9E1A4221B51F226B8845E112
2016-01-14 18:31:36,786 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:36,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 655E94FD9E1A4221B51F226B8845E112
2016-01-14 18:31:36,786 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:36,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:36,787 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:36,787 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796296594_0001_1_00_000000_0
2016-01-14 18:31:36,787 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:36,788 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2d5385c6
2016-01-14 18:31:36,789 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2d5385c6
2016-01-14 18:31:36,789 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:36,789 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:36,790 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:36,790 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 655E94FD9E1A4221B51F226B8845E112
2016-01-14 18:31:36,791 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: B68C4DC7EC83425DB696E280158BD857
2016-01-14 18:31:36,791 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] B68C4DC7EC83425DB696E280158BD857
2016-01-14 18:31:36,792 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@776af729; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@f88e1ff; comparator=cascading.tuple.hadoop.util.TupleComparator@63ad3450; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:36,792 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:36,792 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:36,793 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:36,805 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:36,806 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:36,806 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:36,806 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:36,806 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:36,807 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:36,807 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:36,807 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B33BCEB1D6644C8791D37BCB18E94580, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:36,810 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796296594_0001_1_00_000000_0
2016-01-14 18:31:36,810 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B33BCEB1D6644C8791D37BCB18E94580, mem on close (mb), free: 92, total: 374, max: 672
2016-01-14 18:31:36,810 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:36,811 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:36,811 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:36,811 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:36,812 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:36 Completed running task attempt: attempt_1452796296594_0001_1_00_000000_0
2016-01-14 18:31:36,814 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796296594_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:36,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796296594_0001_1_00, eventCount=1
2016-01-14 18:31:36,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B33BCEB1D6644C8791D37BCB18E94580, taskAttemptId=attempt_1452796296594_0001_1_00_000000_0, startTime=1452796296773, finishTime=1452796296815, timeTaken=42, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796296808, Process_Duration=2, Process_End_Time=1452796296810, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:36,815 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796296594_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:36,815 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:36,815 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796296594_0001_1_00_000000_0
2016-01-14 18:31:36,816 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:36,816 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:36,816 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:TASK_FINISHED]: vertexName=B33BCEB1D6644C8791D37BCB18E94580, taskId=task_1452796296594_0001_1_00_000000, startTime=1452796296773, finishTime=1452796296815, timeTaken=42, status=SUCCEEDED, successfulAttemptID=attempt_1452796296594_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796296808, Process_Duration=2, Process_End_Time=1452796296810, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:36,816 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796296594_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:36,816 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580] : 1
2016-01-14 18:31:36,817 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580]
2016-01-14 18:31:37,096 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:VERTEX_FINISHED]: vertexName=B33BCEB1D6644C8791D37BCB18E94580, vertexId=vertex_1452796296594_0001_1_00, initRequestedTime=1452796296752, initedTime=1452796296763, startRequestedTime=1452796296764, startedTime=1452796296764, finishTime=1452796297096, timeTaken=332, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796296808, Process_Duration=2, Process_End_Time=1452796296810, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796296773, firstTasksToStart=[ task_1452796296594_0001_1_00_000000 ], lastTaskFinishTime=1452796296815, lastTasksToFinish=[ task_1452796296594_0001_1_00_000000 ], minTaskDuration=42, maxTaskDuration=42, avgTaskDuration=42.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796296594_0001_1_00_000000 ], longestDurationTasks=[ task_1452796296594_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:37,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:37,097 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:37,097 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:37,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796296594_0001_00_000001, stoppedTime=1452796297097, exitStatus=0
2016-01-14 18:31:37,097 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:37,097 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937] attempt: attempt_1452796296594_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:37,097 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B1DC95D45A344354B912388F9E7CD937
2016-01-14 18:31:37,097 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937]
2016-01-14 18:31:37,098 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:37,098 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796296594_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:36,817 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=392167424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796296808, Process_Duration=2, Process_End_Time=1452796296810, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:37,098 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:37,098 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:37,098 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796296594_0001_00_000001
2016-01-14 18:31:37,098 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796296594_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:37,098 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796296594_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:37,099 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796296594_0001_00_000001, Exiting
2016-01-14 18:31:37,099 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796296594_0001_00_000001
2016-01-14 18:31:37,099 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796296594_0001_00_000001
2016-01-14 18:31:37,098 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:TASK_STARTED]: vertexName=B1DC95D45A344354B912388F9E7CD937, taskId=task_1452796296594_0001_1_01_000000, scheduledTime=1452796297098, launchTime=1452796297098
2016-01-14 18:31:37,097 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:37,099 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:37,099 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796296594_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:37,099 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937]
2016-01-14 18:31:37,100 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:37,100 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796296594_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:37,100 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: B1DC95D45A344354B912388F9E7CD937, VertexParallelism: 1, TaskAttemptID:attempt_1452796296594_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B33BCEB1D6644C8791D37BCB18E94580, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CB081849CC3A491AA50F22707018164F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:37,100 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796296594_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:37,100 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796296594_0001_00_000001 completed successfully
2016-01-14 18:31:37,100 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:37,101 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:37,101 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796296594_0001_00_000002
2016-01-14 18:31:37,101 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:37,103 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796296594_0001_00_000002
2016-01-14 18:31:37,104 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:37,106 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:37,106 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:37,106 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796296594_0001_00_000002, launchTime=1452796297106
2016-01-14 18:31:37,107 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796296594_0001_00_000002
2016-01-14 18:31:37,107 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796296594_0001_1_01_000000_0] to container: [container_1452796296594_0001_00_000002]
2016-01-14 18:31:37,107 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:37,108 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796296594_0001_00_000002 given task: attempt_1452796296594_0001_1_01_000000_0
2016-01-14 18:31:37,108 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796296594_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796296594_0001_1_01_000000_0
2016-01-14 18:31:37,108 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796296594_0001_1_01_000000_0] started. Is using containerId: [container_1452796296594_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:37 Starting to run new task attempt: attempt_1452796296594_0001_1_01_000000_0
2016-01-14 18:31:37,108 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796296594_0001_1_01_000000_0
2016-01-14 18:31:37,108 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:37,109 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B1DC95D45A344354B912388F9E7CD937, taskAttemptId=attempt_1452796296594_0001_1_01_000000_0, startTime=1452796297108, containerId=container_1452796296594_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796296594_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:37,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:37,109 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796296594_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:37,109 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796296594_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:37,110 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:37,110 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:37,110 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:37,110 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:37,110 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:37,110 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7] every/(1/1) ...lyplatform/0_ARGS_RESULTS, VertexName: B1DC95D45A344354B912388F9E7CD937, VertexParallelism: 1, TaskAttemptID:attempt_1452796296594_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B33BCEB1D6644C8791D37BCB18E94580, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=CB081849CC3A491AA50F22707018164F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:37,110 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:37,111 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796296594_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:37,112 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796296594_0001_1_01_000000_0
2016-01-14 18:31:37,112 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:37,112 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:37,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B33BCEB1D6644C8791D37BCB18E94580, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:37,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:37,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CB081849CC3A491AA50F22707018164F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:37,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B33BCEB1D6644C8791D37BCB18E94580
2016-01-14 18:31:37,114 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:37,113 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:37,115 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:37,115 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:37,115 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B33BCEB1D6644C8791D37BCB18E94580
2016-01-14 18:31:37,116 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B33BCEB1D6644C8791D37BCB18E94580
2016-01-14 18:31:37,117 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:37,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CB081849CC3A491AA50F22707018164F
2016-01-14 18:31:37,117 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:37,117 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: CB081849CC3A491AA50F22707018164F
2016-01-14 18:31:37,122 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B1DC95D45A344354B912388F9E7CD937, ordinal: 1
2016-01-14 18:31:37,122 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1DC95D45A344354B912388F9E7CD937, mem on start (mb), free: 420, total: 561, max: 672
2016-01-14 18:31:37,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:37,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:37,122 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:37,124 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:37,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CB081849CC3A491AA50F22707018164F
2016-01-14 18:31:37,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:37,124 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:37,124 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:37,124 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:37,125 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:37,125 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:37,125 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:37,125 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B33BCEB1D6644C8791D37BCB18E94580, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:37,125 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CB081849CC3A491AA50F22707018164F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:37,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:37,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B33BCEB1D6644C8791D37BCB18E94580 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:37,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:37,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:37,125 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B33BCEB1D6644C8791D37BCB18E94580
2016-01-14 18:31:37,126 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:37,126 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:37,127 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B33BCEB1D6644C8791D37BCB18E94580 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:37,127 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:37,127 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:37,127 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:37,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B33BCEB1D6644C8791D37BCB18E94580
2016-01-14 18:31:37,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:37,128 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796296594_0001_1_01_000000_0
2016-01-14 18:31:37,131 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796296594_0001_1_00_000000_0_10213, runDuration: 32683, ]
2016-01-14 18:31:37,132 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:37,132 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: B68C4DC7EC83425DB696E280158BD857
2016-01-14 18:31:37,132 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS"], id: CB081849CC3A491AA50F22707018164F
2016-01-14 18:31:37,133 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_RESULTS"]
2016-01-14 18:31:37,133 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] B68C4DC7EC83425DB696E280158BD857, for 1 inputs
2016-01-14 18:31:37,133 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796296594_0001_1_00_000000_0_10213] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796297133, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:37,133 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B33BCEB1D6644C8791D37BCB18E94580
2016-01-14 18:31:37,133 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:37,133 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B33BCEB1D6644C8791D37BCB18E94580] #36 in 1ms
2016-01-14 18:31:37,134 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B33BCEB1D6644C8791D37BCB18E94580] #36, status:false, isInterrupted:false
2016-01-14 18:31:37,134 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:37,134 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:37,134 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:37,135 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:37,135 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:37,135 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B33BCEB1D6644C8791D37BCB18E94580
2016-01-14 18:31:37,135 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:37,135 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B1DC95D45A344354B912388F9E7CD937, all 1 inputs ready in: 00:00:00.002
2016-01-14 18:31:37,136 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5a0c4d0f; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5d9741f8
2016-01-14 18:31:37,139 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:37,139 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:37,140 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796296594_0001_1_01_000000_0
2016-01-14 18:31:37,140 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796296594_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:37,141 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796296594_0001_1_01_000000_0
2016-01-14 18:31:37,141 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B1DC95D45A344354B912388F9E7CD937, mem on close (mb), free: 416, total: 561, max: 672
2016-01-14 18:31:37,141 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B33BCEB1D6644C8791D37BCB18E94580
2016-01-14 18:31:37,142 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:37,142 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:37,142 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:37,142 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:37,143 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:37 Completed running task attempt: attempt_1452796296594_0001_1_01_000000_0
2016-01-14 18:31:37,144 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796296594_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:37,144 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:37,144 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B1DC95D45A344354B912388F9E7CD937, taskAttemptId=attempt_1452796296594_0001_1_01_000000_0, startTime=1452796297108, finishTime=1452796297144, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796297135, Process_Duration=6, Process_End_Time=1452796297141, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:37,144 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796296594_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:37,145 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796296594_0001_1_01_000000_0
2016-01-14 18:31:37,145 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:37,145 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:TASK_FINISHED]: vertexName=B1DC95D45A344354B912388F9E7CD937, taskId=task_1452796296594_0001_1_01_000000, startTime=1452796297108, finishTime=1452796297145, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1452796296594_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796297135, Process_Duration=6, Process_End_Time=1452796297141, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:37,145 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796296594_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:37,145 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937] : 1
2016-01-14 18:31:37,145 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:37,145 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937]
2016-01-14 18:31:37,146 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:37,146 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:VERTEX_FINISHED]: vertexName=B1DC95D45A344354B912388F9E7CD937, vertexId=vertex_1452796296594_0001_1_01, initRequestedTime=1452796296753, initedTime=1452796296760, startRequestedTime=1452796296766, startedTime=1452796296766, finishTime=1452796297146, timeTaken=380, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796297135, Process_Duration=6, Process_End_Time=1452796297141, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796297108, firstTasksToStart=[ task_1452796296594_0001_1_01_000000 ], lastTaskFinishTime=1452796297144, lastTasksToFinish=[ task_1452796296594_0001_1_01_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796296594_0001_1_01_000000 ], longestDurationTasks=[ task_1452796296594_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:37,146 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:37,147 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:37,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796297135, Process_Duration=6, Process_End_Time=1452796297141, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:37,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796296594_0001_00_000002, stoppedTime=1452796297147, exitStatus=0
2016-01-14 18:31:37,147 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:37,147 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:37,147 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:37,147 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796296594_0001_00_000002
2016-01-14 18:31:37,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:37,147 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796296594_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:37,147 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:37,148 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796296594_0001_1
2016-01-14 18:31:37,148 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796296594_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:37,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796296594_0001_1
2016-01-14 18:31:37,148 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796296594_0001_00_000002, Exiting
2016-01-14 18:31:37,148 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796296594_0001_00_000002
2016-01-14 18:31:37,148 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796296594_0001_00_000002
2016-01-14 18:31:37,148 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: CB081849CC3A491AA50F22707018164F for vertex: vertex_1452796296594_0001_1_01 [B1DC95D45A344354B912388F9E7CD937]
2016-01-14 18:31:37,148 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796296594_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:37,148 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796296594_0001_00_000002 completed successfully
2016-01-14 18:31:37,155 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796296594_0001_1_00 [B33BCEB1D6644C8791D37BCB18E94580]
2016-01-14 18:31:37,157 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796296594_0001_1, queueSize=0
2016-01-14 18:31:37,157 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:37,158 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796296594_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796296594_0001_1, startTime=1452796296752, finishTime=1452796297155, timeTaken=403, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=980942848, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592593943, Process_Duration=8, Process_End_Time=2905592593951, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:37,158 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796296594_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:37,158 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796296594_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:37,159 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796296594_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:37,159 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796296594_0001_00_000002
2016-01-14 18:31:37 Completed Dag: dag_1452796296594_0001_1
2016-01-14 18:31:37,159 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796296594_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:37,160 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796296594_0001_1_post
2016-01-14 18:31:37,160 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:37,160 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:37,166 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_ARGS_RESULTS, applicationId=application_1452796296594_0001
2016-01-14 18:31:37,167 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:37,167 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:37,168 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:37,168 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:37,168 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:37,168 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:37,169 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:37,169 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:37,170 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:37,170 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:37,170 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:37,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:37,170 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:37,175 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/AA43FA28A8B04871B578628721D3FC34/A52A218B262645DFBDE744262F87A2B7/.tez/application_1452796296594_0001
2016-01-14 18:31:37,176 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.601
2016-01-14 18:31:37,243 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_RESULTS
2016-01-14 18:31:37,259 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:37,259 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:37,267 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:37,268 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:37,295 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 18:31:37,297 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-01-14 18:31:37,298 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:37,298 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:37,299 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_ALL
2016-01-14 18:31:37,320 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:37,321 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:37,352 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-01-14 18:31:37,359 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 18:31:37,360 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:37,361 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:37,361 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:37,376 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:37,381 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE
2016-01-14 18:31:37,427 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:37,428 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:37,428 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE"]
2016-01-14 18:31:37,428 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:37,428 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:37,428 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:37,429 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/0_ARGS_REPLACE
2016-01-14 18:31:37,429 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:37,443 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:37,447 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:37,447 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:37,447 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:37,456 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1/.tez/application_1452796297447_0001 doesn't exist and is created
2016-01-14 18:31:37,471 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:37,471 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:37,472 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1/.tez/application_1452796297447_0001
2016-01-14 18:31:37,474 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796297447_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:37,484 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:37,485 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:37,486 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:37,487 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:37,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1/.tez/application_1452796297447_0001/localmode-log-dir
2016-01-14 18:31:37,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1/.tez/application_1452796297447_0001/localmode-log-dir/history.txt.appattempt_1452796297447_0001_000000, maxErrors=10
2016-01-14 18:31:37,488 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:37,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796297447_0001_000000, appSubmitTime=1452796297473, launchTime=1452796297473
2016-01-14 18:31:37,492 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:35815
2016-01-14 18:31:37,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:37,497 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:37,498 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:37,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:37,499 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:37,500 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796297447_0001_000000, startTime=1452796297500
2016-01-14 18:31:37,500 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:37,500 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:37,571 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:37,572 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:37,572 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1452796297447_0001, dagName=[FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-01-14 18:31:37,588 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-01-14 18:31:37,589 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796297447_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1/.tez/application_1452796297447_0001/localmode-log-dir/dag_1452796297447_0001_1.dot
2016-01-14 18:31:37,590 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1/.tez/application_1452796297447_0001/localmode-log-dir/dag_1452796297447_0001_1-tez-dag.pb.txt
2016-01-14 18:31:37,597 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796297447_0001_1
2016-01-14 18:31:37,597 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:37,597 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-01-14 18:31:37 Running Dag: dag_1452796297447_0001_1
2016-01-14 18:31:37,606 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796297447_0001_1, submitTime=1452796297588
2016-01-14 18:31:37,606 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:37,607 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A46F1976EFFD4420AF1BFE02DEEEE0B7
2016-01-14 18:31:37,607 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:37,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796297447_0001_1, initTime=1452796297607
2016-01-14 18:31:37,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:37,608 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796297447_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:37,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:DAG_STARTED]: dagID=dag_1452796297447_0001_1, startTime=1452796297608
2016-01-14 18:31:37,608 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:37,608 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796297447_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:37,608 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1452796297447_0001, dagName=[FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1] every/(1/1) ...lyplatform/0_ARGS_REPLACE
2016-01-14 18:31:37,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:37,608 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: BACA20B78901411BA7F8D74223C8B616 : {576640CFB6714F51A1A7C70F3307D112={InputName=576640CFB6714F51A1A7C70F3307D112}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:37,609 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 576640CFB6714F51A1A7C70F3307D112, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:37,609 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616]
2016-01-14 18:31:37,609 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616]
2016-01-14 18:31:37,609 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616]
2016-01-14 18:31:37,609 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616]. Starting root input initializers: 1
2016-01-14 18:31:37,610 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:37,610 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7]
2016-01-14 18:31:37,612 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 576640CFB6714F51A1A7C70F3307D112 on vertex vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616]
2016-01-14 18:31:37,613 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:37,613 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7]
2016-01-14 18:31:37,613 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7]
2016-01-14 18:31:37,613 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7]
2016-01-14 18:31:37,614 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=D5CBCB190F07437B9291226FD739D70A, vertexId=vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:37,614 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=D5CBCB190F07437B9291226FD739D70A, vertexId=vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7]
2016-01-14 18:31:37,615 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:37,616 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=D5CBCB190F07437B9291226FD739D70A, vertexName=A46F1976EFFD4420AF1BFE02DEEEE0B7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:37,616 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=D5CBCB190F07437B9291226FD739D70A, vertexId=vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7]
2016-01-14 18:31:37,616 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 576640CFB6714F51A1A7C70F3307D112 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:37,617 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:37,617 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:37,618 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:37,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A46F1976EFFD4420AF1BFE02DEEEE0B7, vertexId=vertex_1452796297447_0001_1_01, initRequestedTime=1452796297610, initedTime=1452796297617, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:37,618 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:37,619 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:37,619 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 576640CFB6714F51A1A7C70F3307D112 on vertex vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616]
2016-01-14 18:31:37,620 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:37,620 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {576640CFB6714F51A1A7C70F3307D112=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:37,620 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616] parallelism set to 1
2016-01-14 18:31:37,620 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:37,621 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796297447_0001_1_00, eventCount=1
2016-01-14 18:31:37,621 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BACA20B78901411BA7F8D74223C8B616, vertexId=vertex_1452796297447_0001_1_00, initRequestedTime=1452796297608, initedTime=1452796297621, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:37,621 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:37,621 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616] with distanceFromRoot: 0
2016-01-14 18:31:37,622 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:37,622 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in BACA20B78901411BA7F8D74223C8B616
2016-01-14 18:31:37,622 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616]
2016-01-14 18:31:37,623 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:37,623 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796297447_0001_1_00, startRequestedTime=1452796297622, startedTime=1452796297622
2016-01-14 18:31:37,623 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:37,624 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:37,624 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:TASK_STARTED]: vertexName=BACA20B78901411BA7F8D74223C8B616, taskId=task_1452796297447_0001_1_00_000000, scheduledTime=1452796297623, launchTime=1452796297623
2016-01-14 18:31:37,624 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796297447_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:37,624 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796297447_0001_1_00 for vertex: vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7] numStartedSources: 1 numSources: 1
2016-01-14 18:31:37,624 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7] with distanceFromRoot: 1
2016-01-14 18:31:37,625 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616]
2016-01-14 18:31:37,625 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: BACA20B78901411BA7F8D74223C8B616 in vertex: A46F1976EFFD4420AF1BFE02DEEEE0B7
2016-01-14 18:31:37,625 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A46F1976EFFD4420AF1BFE02DEEEE0B7 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:37,626 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:37,626 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796297447_0001_1_01, startRequestedTime=1452796297625, startedTime=1452796297625
2016-01-14 18:31:37,626 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:37,626 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: BACA20B78901411BA7F8D74223C8B616, VertexParallelism: 1, TaskAttemptID:attempt_1452796297447_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=576640CFB6714F51A1A7C70F3307D112, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A46F1976EFFD4420AF1BFE02DEEEE0B7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:37,627 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796297447_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:37,627 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796297447_0001, with dag id: dag_1452796297447_0001_1
2016-01-14 18:31:37,627 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:37,628 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:37,628 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:37,628 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:37,629 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796297447_0001_00_000001
2016-01-14 18:31:37,629 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:37,630 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796297447_0001_00_000001
2016-01-14 18:31:37,630 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:37,631 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:37,631 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796297447_0001_00_000001, launchTime=1452796297631
2016-01-14 18:31:37,631 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796297447_0001_00_000001
2016-01-14 18:31:37,632 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:37,632 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796297447_0001_1_00_000000_0] to container: [container_1452796297447_0001_00_000001]
2016-01-14 18:31:37,632 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:37,632 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796297447_0001_00_000001 given task: attempt_1452796297447_0001_1_00_000000_0
2016-01-14 18:31:37,632 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796297447_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796297447_0001_1_00_000000_0
2016-01-14 18:31:37 Starting to run new task attempt: attempt_1452796297447_0001_1_00_000000_0
2016-01-14 18:31:37,633 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796297447_0001_1_00_000000_0
2016-01-14 18:31:37,633 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:37,633 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:37,633 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:37,633 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:37,633 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:37,633 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:37,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: BACA20B78901411BA7F8D74223C8B616, VertexParallelism: 1, TaskAttemptID:attempt_1452796297447_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=576640CFB6714F51A1A7C70F3307D112, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A46F1976EFFD4420AF1BFE02DEEEE0B7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:37,634 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:37,633 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796297447_0001_1_00_000000_0] started. Is using containerId: [container_1452796297447_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:37,634 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:37,634 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BACA20B78901411BA7F8D74223C8B616, taskAttemptId=attempt_1452796297447_0001_1_00_000000_0, startTime=1452796297633, containerId=container_1452796297447_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796297447_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:37,634 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796297447_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:37,634 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796297447_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:37,635 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796297447_0001_1_00_000000_0
2016-01-14 18:31:37,635 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796297447_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:37,635 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:37,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:37,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=576640CFB6714F51A1A7C70F3307D112, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:37,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A46F1976EFFD4420AF1BFE02DEEEE0B7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:37,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:37,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A46F1976EFFD4420AF1BFE02DEEEE0B7
2016-01-14 18:31:37,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:37,637 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:37,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 576640CFB6714F51A1A7C70F3307D112
2016-01-14 18:31:37,638 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 576640CFB6714F51A1A7C70F3307D112
2016-01-14 18:31:37,639 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:37,640 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A46F1976EFFD4420AF1BFE02DEEEE0B7
2016-01-14 18:31:37,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A46F1976EFFD4420AF1BFE02DEEEE0B7
2016-01-14 18:31:37,640 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:37,640 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 576640CFB6714F51A1A7C70F3307D112
2016-01-14 18:31:37,640 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 576640CFB6714F51A1A7C70F3307D112
2016-01-14 18:31:37,641 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:37,644 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BACA20B78901411BA7F8D74223C8B616, ordinal: 0
2016-01-14 18:31:37,644 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BACA20B78901411BA7F8D74223C8B616, mem on start (mb), free: 281, total: 561, max: 672
2016-01-14 18:31:37,644 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:37,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:37,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:37,645 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:37,646 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:37,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:37,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:37,646 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:37,647 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:37,647 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:37,647 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 576640CFB6714F51A1A7C70F3307D112, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:37,647 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A46F1976EFFD4420AF1BFE02DEEEE0B7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:37,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:37,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 576640CFB6714F51A1A7C70F3307D112 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:37,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:37,649 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:37,648 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 576640CFB6714F51A1A7C70F3307D112
2016-01-14 18:31:37,649 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:37,649 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 576640CFB6714F51A1A7C70F3307D112
2016-01-14 18:31:37,649 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:37,650 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796297447_0001_1_00_000000_0
2016-01-14 18:31:37,651 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:37,652 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5a3770d2
2016-01-14 18:31:37,653 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5a3770d2
2016-01-14 18:31:37,654 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:37,654 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:37,653 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:37,655 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 576640CFB6714F51A1A7C70F3307D112
2016-01-14 18:31:37,655 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: FC099C690A2C4E87BCAABC81845567B5
2016-01-14 18:31:37,655 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] FC099C690A2C4E87BCAABC81845567B5
2016-01-14 18:31:37,655 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2b120064; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2ad7bb53; comparator=cascading.tuple.hadoop.util.TupleComparator@394b5c1e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:37,655 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:37,656 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:37,656 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:37,676 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:37,676 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:37,676 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:37,676 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:37,677 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:37,678 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:37,678 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:37,679 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BACA20B78901411BA7F8D74223C8B616, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:37,682 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796297447_0001_1_00_000000_0
2016-01-14 18:31:37,682 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BACA20B78901411BA7F8D74223C8B616, mem on close (mb), free: 322, total: 562, max: 672
2016-01-14 18:31:37,682 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:37,682 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:37,682 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:37,682 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:37,683 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:37 Completed running task attempt: attempt_1452796297447_0001_1_00_000000_0
2016-01-14 18:31:37,686 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796297447_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:37,686 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796297447_0001_1_00, eventCount=1
2016-01-14 18:31:37,686 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BACA20B78901411BA7F8D74223C8B616, taskAttemptId=attempt_1452796297447_0001_1_00_000000_0, startTime=1452796297633, finishTime=1452796297686, timeTaken=53, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796297679, Process_Duration=3, Process_End_Time=1452796297682, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:37,686 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796297447_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:37,686 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796297447_0001_1_00_000000_0
2016-01-14 18:31:37,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:37,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:TASK_FINISHED]: vertexName=BACA20B78901411BA7F8D74223C8B616, taskId=task_1452796297447_0001_1_00_000000, startTime=1452796297633, finishTime=1452796297687, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452796297447_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796297679, Process_Duration=3, Process_End_Time=1452796297682, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:37,687 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796297447_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:37,687 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616] : 1
2016-01-14 18:31:37,687 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616]
2016-01-14 18:31:37,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:37,687 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:VERTEX_FINISHED]: vertexName=BACA20B78901411BA7F8D74223C8B616, vertexId=vertex_1452796297447_0001_1_00, initRequestedTime=1452796297608, initedTime=1452796297621, startRequestedTime=1452796297622, startedTime=1452796297622, finishTime=1452796297687, timeTaken=65, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796297679, Process_Duration=3, Process_End_Time=1452796297682, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796297633, firstTasksToStart=[ task_1452796297447_0001_1_00_000000 ], lastTaskFinishTime=1452796297686, lastTasksToFinish=[ task_1452796297447_0001_1_00_000000 ], minTaskDuration=53, maxTaskDuration=53, avgTaskDuration=53.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796297447_0001_1_00_000000 ], longestDurationTasks=[ task_1452796297447_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:37,687 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:37,688 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:37,688 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7] attempt: attempt_1452796297447_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:37,688 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A46F1976EFFD4420AF1BFE02DEEEE0B7
2016-01-14 18:31:37,688 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7]
2016-01-14 18:31:37,688 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:37,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:TASK_STARTED]: vertexName=A46F1976EFFD4420AF1BFE02DEEEE0B7, taskId=task_1452796297447_0001_1_01_000000, scheduledTime=1452796297688, launchTime=1452796297688
2016-01-14 18:31:37,688 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796297447_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:37,688 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7]
2016-01-14 18:31:37,688 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:37,688 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:37,688 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: A46F1976EFFD4420AF1BFE02DEEEE0B7, VertexParallelism: 1, TaskAttemptID:attempt_1452796297447_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BACA20B78901411BA7F8D74223C8B616, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D5CBCB190F07437B9291226FD739D70A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:37,689 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796297679, Process_Duration=3, Process_End_Time=1452796297682, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:37,689 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:37,689 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:37,691 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796297447_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:37,692 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:37,692 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796297447_0001_00_000001, stoppedTime=1452796297692, exitStatus=0
2016-01-14 18:31:37,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:37,692 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:37,692 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:37,692 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796297447_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:37,692 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796297447_0001_00_000002
2016-01-14 18:31:37,693 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:37,693 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:37,693 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796297447_0001_00_000001
2016-01-14 18:31:37,693 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796297447_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:37,694 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796297447_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:37,694 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796297447_0001_00_000001, Exiting
2016-01-14 18:31:37,694 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796297447_0001_00_000001
2016-01-14 18:31:37,694 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796297447_0001_00_000001
2016-01-14 18:31:37,695 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796297447_0001_00_000002
2016-01-14 18:31:37,696 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:37,698 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796297447_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:37,698 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796297447_0001_00_000001 completed successfully
2016-01-14 18:31:37,698 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:37,699 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:37,699 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:37,699 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796297447_0001_00_000002, launchTime=1452796297699
2016-01-14 18:31:37,700 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796297447_0001_00_000002
2016-01-14 18:31:37,700 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796297447_0001_1_01_000000_0] to container: [container_1452796297447_0001_00_000002]
2016-01-14 18:31:37,700 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:37,700 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796297447_0001_00_000002 given task: attempt_1452796297447_0001_1_01_000000_0
2016-01-14 18:31:37,701 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796297447_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796297447_0001_1_01_000000_0
2016-01-14 18:31:37,701 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796297447_0001_1_01_000000_0] started. Is using containerId: [container_1452796297447_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:37,701 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:37,701 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A46F1976EFFD4420AF1BFE02DEEEE0B7, taskAttemptId=attempt_1452796297447_0001_1_01_000000_0, startTime=1452796297700, containerId=container_1452796297447_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796297447_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:37,701 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796297447_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:37 Starting to run new task attempt: attempt_1452796297447_0001_1_01_000000_0
2016-01-14 18:31:37,701 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796297447_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:37,701 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796297447_0001_1_01_000000_0
2016-01-14 18:31:37,701 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:37,701 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:37,701 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:37,701 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:37,701 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:37,702 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:37,702 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1] every/(1/1) ...lyplatform/0_ARGS_REPLACE, VertexName: A46F1976EFFD4420AF1BFE02DEEEE0B7, VertexParallelism: 1, TaskAttemptID:attempt_1452796297447_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BACA20B78901411BA7F8D74223C8B616, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=D5CBCB190F07437B9291226FD739D70A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:37,702 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:37,702 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796297447_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:37,703 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796297447_0001_1_01_000000_0
2016-01-14 18:31:37,703 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:37,703 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:37,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:37,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BACA20B78901411BA7F8D74223C8B616, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:37,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:37,704 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BACA20B78901411BA7F8D74223C8B616
2016-01-14 18:31:37,706 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:37,706 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D5CBCB190F07437B9291226FD739D70A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:37,706 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:37,706 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:37,706 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: BACA20B78901411BA7F8D74223C8B616
2016-01-14 18:31:37,707 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D5CBCB190F07437B9291226FD739D70A
2016-01-14 18:31:37,707 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:37,706 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:37,707 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: D5CBCB190F07437B9291226FD739D70A
2016-01-14 18:31:37,707 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BACA20B78901411BA7F8D74223C8B616
2016-01-14 18:31:37,710 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A46F1976EFFD4420AF1BFE02DEEEE0B7, ordinal: 1
2016-01-14 18:31:37,710 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A46F1976EFFD4420AF1BFE02DEEEE0B7, mem on start (mb), free: 315, total: 562, max: 672
2016-01-14 18:31:37,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:37,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:37,710 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:37,714 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:37,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D5CBCB190F07437B9291226FD739D70A
2016-01-14 18:31:37,715 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:37,715 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:37,715 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:37,715 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:37,715 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:37,715 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:37,715 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:37,715 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BACA20B78901411BA7F8D74223C8B616, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:37,716 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D5CBCB190F07437B9291226FD739D70A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:37,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:37,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BACA20B78901411BA7F8D74223C8B616 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:37,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:37,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:37,716 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BACA20B78901411BA7F8D74223C8B616
2016-01-14 18:31:37,716 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:37,717 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:37,717 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: BACA20B78901411BA7F8D74223C8B616 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:37,717 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:37,717 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:37,717 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:37,724 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BACA20B78901411BA7F8D74223C8B616
2016-01-14 18:31:37,725 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:37,726 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796297447_0001_1_01_000000_0
2016-01-14 18:31:37,728 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:37,729 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: FC099C690A2C4E87BCAABC81845567B5
2016-01-14 18:31:37,729 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE"], id: D5CBCB190F07437B9291226FD739D70A
2016-01-14 18:31:37,729 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_REPLACE"]
2016-01-14 18:31:37,729 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] FC099C690A2C4E87BCAABC81845567B5, for 1 inputs
2016-01-14 18:31:37,729 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796297447_0001_1_00_000000_0_10219, runDuration: 47138, ]
2016-01-14 18:31:37,730 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796297447_0001_1_00_000000_0_10219] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796297730, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:37,731 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : BACA20B78901411BA7F8D74223C8B616
2016-01-14 18:31:37,731 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:37,731 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [BACA20B78901411BA7F8D74223C8B616] #37 in 1ms
2016-01-14 18:31:37,731 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [BACA20B78901411BA7F8D74223C8B616] #37, status:false, isInterrupted:false
2016-01-14 18:31:37,732 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:37,732 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:37,732 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:37,732 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:37,732 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:37,732 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : BACA20B78901411BA7F8D74223C8B616
2016-01-14 18:31:37,732 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:37,733 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A46F1976EFFD4420AF1BFE02DEEEE0B7, all 1 inputs ready in: 00:00:00.004
2016-01-14 18:31:37,733 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@228063fd; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1b2488bd
2016-01-14 18:31:37,736 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:37,736 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:37,737 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796297447_0001_1_01_000000_0
2016-01-14 18:31:37,737 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796297447_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:37,738 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796297447_0001_1_01_000000_0
2016-01-14 18:31:37,738 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A46F1976EFFD4420AF1BFE02DEEEE0B7, mem on close (mb), free: 312, total: 562, max: 672
2016-01-14 18:31:37,738 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: BACA20B78901411BA7F8D74223C8B616
2016-01-14 18:31:37,738 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:37,739 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:37,739 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:37,739 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:37,739 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:37 Completed running task attempt: attempt_1452796297447_0001_1_01_000000_0
2016-01-14 18:31:37,740 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796297447_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:37,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:37,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A46F1976EFFD4420AF1BFE02DEEEE0B7, taskAttemptId=attempt_1452796297447_0001_1_01_000000_0, startTime=1452796297700, finishTime=1452796297740, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796297733, Process_Duration=5, Process_End_Time=1452796297738, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:37,741 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796297447_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:37,741 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:37,741 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796297447_0001_1_01_000000_0
2016-01-14 18:31:37,741 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:TASK_FINISHED]: vertexName=A46F1976EFFD4420AF1BFE02DEEEE0B7, taskId=task_1452796297447_0001_1_01_000000, startTime=1452796297700, finishTime=1452796297741, timeTaken=41, status=SUCCEEDED, successfulAttemptID=attempt_1452796297447_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796297733, Process_Duration=5, Process_End_Time=1452796297738, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:37,742 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796297447_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:37,742 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7] : 1
2016-01-14 18:31:37,742 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7]
2016-01-14 18:31:37,743 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:VERTEX_FINISHED]: vertexName=A46F1976EFFD4420AF1BFE02DEEEE0B7, vertexId=vertex_1452796297447_0001_1_01, initRequestedTime=1452796297610, initedTime=1452796297617, startRequestedTime=1452796297625, startedTime=1452796297625, finishTime=1452796297742, timeTaken=117, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796297733, Process_Duration=5, Process_End_Time=1452796297738, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796297700, firstTasksToStart=[ task_1452796297447_0001_1_01_000000 ], lastTaskFinishTime=1452796297740, lastTasksToFinish=[ task_1452796297447_0001_1_01_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796297447_0001_1_01_000000 ], longestDurationTasks=[ task_1452796297447_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:37,743 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:37,743 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:37,744 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:37,744 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:37,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796297447_0001_00_000002, stoppedTime=1452796297744, exitStatus=0
2016-01-14 18:31:37,744 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:37,744 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:37,744 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796297447_0001_1
2016-01-14 18:31:37,744 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796297447_0001_1
2016-01-14 18:31:37,745 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796297447_0001_1_00 [BACA20B78901411BA7F8D74223C8B616]
2016-01-14 18:31:37,745 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: D5CBCB190F07437B9291226FD739D70A for vertex: vertex_1452796297447_0001_1_01 [A46F1976EFFD4420AF1BFE02DEEEE0B7]
2016-01-14 18:31:37,744 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:37,748 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=183, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796297733, Process_Duration=5, Process_End_Time=1452796297738, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:37,748 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:37,748 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:37,749 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796297447_0001_00_000002
2016-01-14 18:31:37,749 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796297447_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:37,749 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796297447_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:37,749 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796297447_0001_00_000002, Exiting
2016-01-14 18:31:37,749 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796297447_0001_00_000002
2016-01-14 18:31:37,749 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796297447_0001_00_000002
2016-01-14 18:31:37,750 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796297447_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:37,750 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796297447_0001_00_000002 completed successfully
2016-01-14 18:31:37,756 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796297447_0001_1, queueSize=0
2016-01-14 18:31:37,756 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796297447_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796297447_0001_1, startTime=1452796297608, finishTime=1452796297754, timeTaken=146, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1036, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1178599424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=14, MERGE_PHASE_TIME=15, FIRST_EVENT_RECEIVED=15, LAST_EVENT_RECEIVED=15, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592595412, Process_Duration=8, Process_End_Time=2905592595420, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:37,757 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796297447_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:37,757 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796297447_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:37,757 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796297447_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:37,757 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796297447_0001_00_000002
2016-01-14 18:31:37 Completed Dag: dag_1452796297447_0001_1
2016-01-14 18:31:37,757 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796297447_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:37,757 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796297447_0001_1_post
2016-01-14 18:31:37,757 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:37,757 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:37,758 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:37,765 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/0_ARGS_REPLACE, applicationId=application_1452796297447_0001
2016-01-14 18:31:37,767 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:37,767 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:37,767 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:37,767 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:37,767 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:37,771 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:37,771 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:37,772 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:37,773 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:37,773 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:37,773 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:37,773 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:37,773 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:37,773 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:37,774 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:37,779 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FEEE9615BDDB4D54BD2A1C49D6BC1E3C/FFE427B385724B1C945A9088CAA189E1/.tez/application_1452796297447_0001
2016-01-14 18:31:37,779 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.352
2016-01-14 18:31:37,873 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_REPLACE
2016-01-14 18:31:37,896 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:37,896 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:37,929 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 18:31:37,937 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 18:31:37,937 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:37,938 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:37,939 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:37,955 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:37,960 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP
2016-01-14 18:31:38,008 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:38,008 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:38,008 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP"]
2016-01-14 18:31:38,008 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:38,008 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:38,008 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:38,010 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...emblyplatform/0_ARGS_SWAP
2016-01-14 18:31:38,011 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:38,023 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:38,027 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:38,027 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:38,028 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:38,036 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF/.tez/application_1452796298027_0001 doesn't exist and is created
2016-01-14 18:31:38,050 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:38,050 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:38,050 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF/.tez/application_1452796298027_0001
2016-01-14 18:31:38,052 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796298027_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:38,063 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:38,063 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:38,064 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:38,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:38,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF/.tez/application_1452796298027_0001/localmode-log-dir
2016-01-14 18:31:38,065 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF/.tez/application_1452796298027_0001/localmode-log-dir/history.txt.appattempt_1452796298027_0001_000000, maxErrors=10
2016-01-14 18:31:38,065 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:38,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796298027_0001_000000, appSubmitTime=1452796298052, launchTime=1452796298052
2016-01-14 18:31:38,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:38,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:38,070 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:38,070 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:38,074 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:56830
2016-01-14 18:31:38,075 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:38,076 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796298027_0001_000000, startTime=1452796298076
2016-01-14 18:31:38,076 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:38,077 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:38,150 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:38,150 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:38,150 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1452796298027_0001, dagName=[EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-01-14 18:31:38,166 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-01-14 18:31:38,167 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796298027_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF/.tez/application_1452796298027_0001/localmode-log-dir/dag_1452796298027_0001_1.dot
2016-01-14 18:31:38,168 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF/.tez/application_1452796298027_0001/localmode-log-dir/dag_1452796298027_0001_1-tez-dag.pb.txt
2016-01-14 18:31:38,174 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796298027_0001_1
2016-01-14 18:31:38,175 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,175 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-01-14 18:31:38 Running Dag: dag_1452796298027_0001_1
2016-01-14 18:31:38,185 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796298027_0001_1, submitTime=1452796298166
2016-01-14 18:31:38,185 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:38,186 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F11EC1D36B9244E9BD8889EF1D65E2E8
2016-01-14 18:31:38,186 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:38,187 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796298027_0001_1, initTime=1452796298186
2016-01-14 18:31:38,187 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:38,187 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796298027_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:38,189 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:DAG_STARTED]: dagID=dag_1452796298027_0001_1, startTime=1452796298189
2016-01-14 18:31:38,189 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:38,189 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796298027_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:38,189 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:38,189 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A5EC389D43FF4B96A2F2311F478B80B0 : {121EA5207D78411B8238CB4F06B3C265={InputName=121EA5207D78411B8238CB4F06B3C265}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:38,189 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1452796298027_0001, dagName=[EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF] every/(1/1) ...emblyplatform/0_ARGS_SWAP
2016-01-14 18:31:38,189 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 121EA5207D78411B8238CB4F06B3C265, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:38,190 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0]
2016-01-14 18:31:38,190 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0]
2016-01-14 18:31:38,190 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0]
2016-01-14 18:31:38,190 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0]. Starting root input initializers: 1
2016-01-14 18:31:38,190 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:38,190 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8]
2016-01-14 18:31:38,191 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 121EA5207D78411B8238CB4F06B3C265 on vertex vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0]
2016-01-14 18:31:38,193 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:38,193 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 121EA5207D78411B8238CB4F06B3C265 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:38,193 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:38,193 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:38,193 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:38,194 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8]
2016-01-14 18:31:38,194 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8]
2016-01-14 18:31:38,194 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8]
2016-01-14 18:31:38,194 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3ECFA396D195456A8E92E937C3FAD723, vertexId=vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:38,194 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3ECFA396D195456A8E92E937C3FAD723, vertexId=vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8]
2016-01-14 18:31:38,195 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:38,195 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 121EA5207D78411B8238CB4F06B3C265 on vertex vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0]
2016-01-14 18:31:38,196 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3ECFA396D195456A8E92E937C3FAD723, vertexName=F11EC1D36B9244E9BD8889EF1D65E2E8, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:38,196 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3ECFA396D195456A8E92E937C3FAD723, vertexId=vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8]
2016-01-14 18:31:38,197 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:38,197 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F11EC1D36B9244E9BD8889EF1D65E2E8, vertexId=vertex_1452796298027_0001_1_01, initRequestedTime=1452796298190, initedTime=1452796298197, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:38,197 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:38,197 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:38,198 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {121EA5207D78411B8238CB4F06B3C265=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:38,198 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0] parallelism set to 1
2016-01-14 18:31:38,198 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:38,198 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796298027_0001_1_00, eventCount=1
2016-01-14 18:31:38,198 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A5EC389D43FF4B96A2F2311F478B80B0, vertexId=vertex_1452796298027_0001_1_00, initRequestedTime=1452796298189, initedTime=1452796298198, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:38,199 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0] with distanceFromRoot: 0
2016-01-14 18:31:38,199 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:38,199 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A5EC389D43FF4B96A2F2311F478B80B0
2016-01-14 18:31:38,199 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0]
2016-01-14 18:31:38,199 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796298027_0001_1_00, startRequestedTime=1452796298199, startedTime=1452796298199
2016-01-14 18:31:38,200 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:38,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:38,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:38,200 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:38,200 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:TASK_STARTED]: vertexName=A5EC389D43FF4B96A2F2311F478B80B0, taskId=task_1452796298027_0001_1_00_000000, scheduledTime=1452796298200, launchTime=1452796298200
2016-01-14 18:31:38,200 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298027_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:38,201 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796298027_0001_1_00 for vertex: vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8] numStartedSources: 1 numSources: 1
2016-01-14 18:31:38,201 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8] with distanceFromRoot: 1
2016-01-14 18:31:38,201 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0]
2016-01-14 18:31:38,201 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A5EC389D43FF4B96A2F2311F478B80B0 in vertex: F11EC1D36B9244E9BD8889EF1D65E2E8
2016-01-14 18:31:38,201 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F11EC1D36B9244E9BD8889EF1D65E2E8 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:38,202 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796298027_0001_1_01, startRequestedTime=1452796298201, startedTime=1452796298201
2016-01-14 18:31:38,202 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:38,202 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:38,202 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: A5EC389D43FF4B96A2F2311F478B80B0, VertexParallelism: 1, TaskAttemptID:attempt_1452796298027_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=121EA5207D78411B8238CB4F06B3C265, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F11EC1D36B9244E9BD8889EF1D65E2E8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:38,202 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298027_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:38,203 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:38,203 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:38,204 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:38,204 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:38,204 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796298027_0001_00_000001
2016-01-14 18:31:38,204 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:38,206 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796298027_0001_00_000001
2016-01-14 18:31:38,206 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:38,207 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:38,207 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:38,207 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796298027_0001_00_000001, launchTime=1452796298207
2016-01-14 18:31:38,208 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796298027_0001_00_000001
2016-01-14 18:31:38,208 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796298027_0001_1_00_000000_0] to container: [container_1452796298027_0001_00_000001]
2016-01-14 18:31:38,208 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:38,209 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796298027_0001_1_00_000000_0] started. Is using containerId: [container_1452796298027_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:38,209 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:38,209 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A5EC389D43FF4B96A2F2311F478B80B0, taskAttemptId=attempt_1452796298027_0001_1_00_000000_0, startTime=1452796298209, containerId=container_1452796298027_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796298027_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:38,209 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298027_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:38,210 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298027_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:38,210 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796298027_0001_00_000001 given task: attempt_1452796298027_0001_1_00_000000_0
2016-01-14 18:31:38,210 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796298027_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796298027_0001_1_00_000000_0
2016-01-14 18:31:38 Starting to run new task attempt: attempt_1452796298027_0001_1_00_000000_0
2016-01-14 18:31:38,210 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796298027_0001_1_00_000000_0
2016-01-14 18:31:38,211 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,211 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:38,211 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:38,211 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:38,211 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:38,211 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:38,211 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: A5EC389D43FF4B96A2F2311F478B80B0, VertexParallelism: 1, TaskAttemptID:attempt_1452796298027_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=121EA5207D78411B8238CB4F06B3C265, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F11EC1D36B9244E9BD8889EF1D65E2E8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:38,211 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:38,212 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796298027_0001_1_00_000000_0
2016-01-14 18:31:38,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:38,212 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,213 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796298027_0001, with dag id: dag_1452796298027_0001_1
2016-01-14 18:31:38,213 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796298027_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:38,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=121EA5207D78411B8238CB4F06B3C265, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:38,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:38,214 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 121EA5207D78411B8238CB4F06B3C265
2016-01-14 18:31:38,214 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 121EA5207D78411B8238CB4F06B3C265
2016-01-14 18:31:38,215 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F11EC1D36B9244E9BD8889EF1D65E2E8, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:38,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:38,216 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F11EC1D36B9244E9BD8889EF1D65E2E8
2016-01-14 18:31:38,218 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:38,218 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:38,218 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F11EC1D36B9244E9BD8889EF1D65E2E8
2016-01-14 18:31:38,221 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F11EC1D36B9244E9BD8889EF1D65E2E8
2016-01-14 18:31:38,222 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:38,222 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A5EC389D43FF4B96A2F2311F478B80B0, ordinal: 0
2016-01-14 18:31:38,223 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5EC389D43FF4B96A2F2311F478B80B0, mem on start (mb), free: 201, total: 562, max: 672
2016-01-14 18:31:38,223 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,223 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:38,223 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:38,223 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 121EA5207D78411B8238CB4F06B3C265
2016-01-14 18:31:38,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 121EA5207D78411B8238CB4F06B3C265
2016-01-14 18:31:38,224 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:38,224 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:38,224 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:38,224 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:38,225 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:38,225 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:38,225 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:38,225 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 121EA5207D78411B8238CB4F06B3C265, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:38,225 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F11EC1D36B9244E9BD8889EF1D65E2E8, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:38,225 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:38,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 121EA5207D78411B8238CB4F06B3C265
2016-01-14 18:31:38,226 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:38,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 121EA5207D78411B8238CB4F06B3C265
2016-01-14 18:31:38,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 121EA5207D78411B8238CB4F06B3C265 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:38,226 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:38,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:38,227 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:38,227 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796298027_0001_1_00_000000_0
2016-01-14 18:31:38,228 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:38,229 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3a5375db
2016-01-14 18:31:38,230 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3a5375db
2016-01-14 18:31:38,230 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:38,231 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:38,231 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:38,233 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 121EA5207D78411B8238CB4F06B3C265
2016-01-14 18:31:38,233 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 183D95F1BD794BF48BBFAF8FB2B1343C
2016-01-14 18:31:38,233 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 183D95F1BD794BF48BBFAF8FB2B1343C
2016-01-14 18:31:38,234 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@74731357; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@2efaf07b; comparator=cascading.tuple.hadoop.util.TupleComparator@5cc23355; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:38,234 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:38,234 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:38,234 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:38,257 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:38,257 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:38,257 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:38,257 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:38,257 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:38,259 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:38,259 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:38,259 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A5EC389D43FF4B96A2F2311F478B80B0, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:38,263 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796298027_0001_1_00_000000_0
2016-01-14 18:31:38,263 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A5EC389D43FF4B96A2F2311F478B80B0, mem on close (mb), free: 321, total: 561, max: 672
2016-01-14 18:31:38,263 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:38,263 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:38,263 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:38,264 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:38,265 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:38 Completed running task attempt: attempt_1452796298027_0001_1_00_000000_0
2016-01-14 18:31:38,268 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796298027_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:38,269 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796298259, Process_Duration=4, Process_End_Time=1452796298263, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:38,269 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796298027_0001_1_00, eventCount=1
2016-01-14 18:31:38,270 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A5EC389D43FF4B96A2F2311F478B80B0, taskAttemptId=attempt_1452796298027_0001_1_00_000000_0, startTime=1452796298209, finishTime=1452796298269, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298259, Process_Duration=4, Process_End_Time=1452796298263, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:38,270 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298027_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:38,270 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796298027_0001_1_00_000000_0
2016-01-14 18:31:38,270 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:38,271 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:TASK_FINISHED]: vertexName=A5EC389D43FF4B96A2F2311F478B80B0, taskId=task_1452796298027_0001_1_00_000000, startTime=1452796298209, finishTime=1452796298270, timeTaken=61, status=SUCCEEDED, successfulAttemptID=attempt_1452796298027_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298259, Process_Duration=4, Process_End_Time=1452796298263, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:38,271 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298027_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:38,271 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:38,271 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0] : 1
2016-01-14 18:31:38,271 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0]
2016-01-14 18:31:38,272 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:38,272 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:VERTEX_FINISHED]: vertexName=A5EC389D43FF4B96A2F2311F478B80B0, vertexId=vertex_1452796298027_0001_1_00, initRequestedTime=1452796298189, initedTime=1452796298198, startRequestedTime=1452796298199, startedTime=1452796298199, finishTime=1452796298271, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298259, Process_Duration=4, Process_End_Time=1452796298263, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796298209, firstTasksToStart=[ task_1452796298027_0001_1_00_000000 ], lastTaskFinishTime=1452796298269, lastTasksToFinish=[ task_1452796298027_0001_1_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796298027_0001_1_00_000000 ], longestDurationTasks=[ task_1452796298027_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:38,272 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:38,272 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:38,272 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:38,272 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796298027_0001_00_000001, stoppedTime=1452796298272, exitStatus=0
2016-01-14 18:31:38,272 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:38,273 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8] attempt: attempt_1452796298027_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:38,273 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F11EC1D36B9244E9BD8889EF1D65E2E8
2016-01-14 18:31:38,273 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:38,273 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:38,273 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,273 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796298027_0001_00_000001
2016-01-14 18:31:38,273 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796298027_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:38,273 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796298027_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:38,273 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796298027_0001_00_000001, Exiting
2016-01-14 18:31:38,274 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796298027_0001_00_000001
2016-01-14 18:31:38,274 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796298027_0001_00_000001
2016-01-14 18:31:38,273 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8]
2016-01-14 18:31:38,275 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:38,275 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796298027_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:38,275 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:TASK_STARTED]: vertexName=F11EC1D36B9244E9BD8889EF1D65E2E8, taskId=task_1452796298027_0001_1_01_000000, scheduledTime=1452796298275, launchTime=1452796298275
2016-01-14 18:31:38,275 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298027_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:38,276 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:38,276 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8]
2016-01-14 18:31:38,276 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: F11EC1D36B9244E9BD8889EF1D65E2E8, VertexParallelism: 1, TaskAttemptID:attempt_1452796298027_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A5EC389D43FF4B96A2F2311F478B80B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3ECFA396D195456A8E92E937C3FAD723, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:38,275 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:38,277 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298027_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:38,277 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:38,277 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796298027_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:38,277 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796298027_0001_00_000001 completed successfully
2016-01-14 18:31:38,278 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796298027_0001_00_000002
2016-01-14 18:31:38,278 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:38,278 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:38,279 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796298027_0001_00_000002
2016-01-14 18:31:38,280 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:38,282 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:38,283 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796298027_0001_00_000002, launchTime=1452796298282
2016-01-14 18:31:38,283 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796298027_0001_00_000002
2016-01-14 18:31:38,283 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:38,284 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796298027_0001_1_01_000000_0] to container: [container_1452796298027_0001_00_000002]
2016-01-14 18:31:38,284 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:38,284 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796298027_0001_00_000002 given task: attempt_1452796298027_0001_1_01_000000_0
2016-01-14 18:31:38,284 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796298027_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796298027_0001_1_01_000000_0
2016-01-14 18:31:38,284 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796298027_0001_1_01_000000_0] started. Is using containerId: [container_1452796298027_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:38 Starting to run new task attempt: attempt_1452796298027_0001_1_01_000000_0
2016-01-14 18:31:38,284 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796298027_0001_1_01_000000_0
2016-01-14 18:31:38,284 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,284 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:38,285 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:38,285 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:38,285 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:38,285 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:38,285 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:38,285 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F11EC1D36B9244E9BD8889EF1D65E2E8, taskAttemptId=attempt_1452796298027_0001_1_01_000000_0, startTime=1452796298284, containerId=container_1452796298027_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796298027_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:38,285 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF] every/(1/1) ...emblyplatform/0_ARGS_SWAP, VertexName: F11EC1D36B9244E9BD8889EF1D65E2E8, VertexParallelism: 1, TaskAttemptID:attempt_1452796298027_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A5EC389D43FF4B96A2F2311F478B80B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3ECFA396D195456A8E92E937C3FAD723, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:38,285 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:38,286 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298027_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:38,286 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298027_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:38,286 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796298027_0001_1_01_000000_0
2016-01-14 18:31:38,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:38,286 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,287 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A5EC389D43FF4B96A2F2311F478B80B0, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:38,287 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796298027_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:38,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:38,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A5EC389D43FF4B96A2F2311F478B80B0
2016-01-14 18:31:38,288 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,289 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:38,290 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:38,290 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A5EC389D43FF4B96A2F2311F478B80B0
2016-01-14 18:31:38,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A5EC389D43FF4B96A2F2311F478B80B0
2016-01-14 18:31:38,290 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3ECFA396D195456A8E92E937C3FAD723, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:38,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:38,291 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3ECFA396D195456A8E92E937C3FAD723
2016-01-14 18:31:38,291 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:38,291 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:38,291 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3ECFA396D195456A8E92E937C3FAD723
2016-01-14 18:31:38,296 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F11EC1D36B9244E9BD8889EF1D65E2E8, ordinal: 1
2016-01-14 18:31:38,296 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F11EC1D36B9244E9BD8889EF1D65E2E8, mem on start (mb), free: 314, total: 561, max: 672
2016-01-14 18:31:38,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,296 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:38,297 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:38,301 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:38,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3ECFA396D195456A8E92E937C3FAD723
2016-01-14 18:31:38,301 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:38,301 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:38,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:38,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:38,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:38,301 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:38,302 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:38,302 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A5EC389D43FF4B96A2F2311F478B80B0, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:38,302 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3ECFA396D195456A8E92E937C3FAD723, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:38,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:38,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A5EC389D43FF4B96A2F2311F478B80B0 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:38,302 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A5EC389D43FF4B96A2F2311F478B80B0
2016-01-14 18:31:38,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:38,303 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:38,303 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:38,303 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:38,303 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A5EC389D43FF4B96A2F2311F478B80B0 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:38,304 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:38,304 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:38,304 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:38,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A5EC389D43FF4B96A2F2311F478B80B0
2016-01-14 18:31:38,306 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:38,307 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796298027_0001_1_01_000000_0
2016-01-14 18:31:38,309 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:38,309 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796298027_0001_1_00_000000_0_10225, runDuration: 50763, ]
2016-01-14 18:31:38,309 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 183D95F1BD794BF48BBFAF8FB2B1343C
2016-01-14 18:31:38,310 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP"], id: 3ECFA396D195456A8E92E937C3FAD723
2016-01-14 18:31:38,310 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/0_ARGS_SWAP"]
2016-01-14 18:31:38,310 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 183D95F1BD794BF48BBFAF8FB2B1343C, for 1 inputs
2016-01-14 18:31:38,311 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796298027_0001_1_00_000000_0_10225] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796298311, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:38,312 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A5EC389D43FF4B96A2F2311F478B80B0
2016-01-14 18:31:38,312 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:38,312 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A5EC389D43FF4B96A2F2311F478B80B0] #38 in 2ms
2016-01-14 18:31:38,312 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A5EC389D43FF4B96A2F2311F478B80B0] #38, status:false, isInterrupted:false
2016-01-14 18:31:38,313 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:38,313 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:38,313 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:38,313 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:38,313 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:38,314 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A5EC389D43FF4B96A2F2311F478B80B0
2016-01-14 18:31:38,314 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:38,314 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F11EC1D36B9244E9BD8889EF1D65E2E8, all 1 inputs ready in: 00:00:00.004
2016-01-14 18:31:38,314 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@59e98cde; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@741f70ef
2016-01-14 18:31:38,316 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:38,316 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:38,318 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:38,318 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:38,318 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796298027_0001_1_01_000000_0
2016-01-14 18:31:38,318 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796298027_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:38,318 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796298027_0001_1_01_000000_0
2016-01-14 18:31:38,318 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F11EC1D36B9244E9BD8889EF1D65E2E8, mem on close (mb), free: 311, total: 561, max: 672
2016-01-14 18:31:38,318 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A5EC389D43FF4B96A2F2311F478B80B0
2016-01-14 18:31:38,319 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:38,319 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:38,319 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:38,319 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:38,319 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:38 Completed running task attempt: attempt_1452796298027_0001_1_01_000000_0
2016-01-14 18:31:38,320 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796298027_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:38,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F11EC1D36B9244E9BD8889EF1D65E2E8, taskAttemptId=attempt_1452796298027_0001_1_01_000000_0, startTime=1452796298284, finishTime=1452796298320, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=703, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298314, Process_Duration=4, Process_End_Time=1452796298318, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:38,321 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298027_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:38,321 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796298027_0001_1_01_000000_0
2016-01-14 18:31:38,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:38,322 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:TASK_FINISHED]: vertexName=F11EC1D36B9244E9BD8889EF1D65E2E8, taskId=task_1452796298027_0001_1_01_000000, startTime=1452796298284, finishTime=1452796298321, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1452796298027_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=703, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298314, Process_Duration=4, Process_End_Time=1452796298318, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:38,322 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:38,322 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298027_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:38,322 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8] : 1
2016-01-14 18:31:38,322 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8]
2016-01-14 18:31:38,322 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:38,323 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:38,323 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:VERTEX_FINISHED]: vertexName=F11EC1D36B9244E9BD8889EF1D65E2E8, vertexId=vertex_1452796298027_0001_1_01, initRequestedTime=1452796298190, initedTime=1452796298197, startRequestedTime=1452796298201, startedTime=1452796298201, finishTime=1452796298322, timeTaken=121, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=703, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298314, Process_Duration=4, Process_End_Time=1452796298318, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796298284, firstTasksToStart=[ task_1452796298027_0001_1_01_000000 ], lastTaskFinishTime=1452796298320, lastTasksToFinish=[ task_1452796298027_0001_1_01_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796298027_0001_1_01_000000 ], longestDurationTasks=[ task_1452796298027_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:38,323 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:38,325 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:38,325 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796298027_0001_00_000002, stoppedTime=1452796298325, exitStatus=0
2016-01-14 18:31:38,325 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:38,325 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:38,325 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:38,326 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796298027_0001_1
2016-01-14 18:31:38,326 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796298027_0001_1
2016-01-14 18:31:38,326 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796298027_0001_1_00 [A5EC389D43FF4B96A2F2311F478B80B0]
2016-01-14 18:31:38,326 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 3ECFA396D195456A8E92E937C3FAD723 for vertex: vertex_1452796298027_0001_1_01 [F11EC1D36B9244E9BD8889EF1D65E2E8]
2016-01-14 18:31:38,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=703, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796298314, Process_Duration=4, Process_End_Time=1452796298318, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:38,328 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:38,328 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,328 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796298027_0001_00_000002
2016-01-14 18:31:38,329 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796298027_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:38,329 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796298027_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:38,329 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796298027_0001_00_000002, Exiting
2016-01-14 18:31:38,329 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796298027_0001_00_000002
2016-01-14 18:31:38,329 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796298027_0001_00_000002
2016-01-14 18:31:38,330 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796298027_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:38,330 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796298027_0001_00_000002 completed successfully
2016-01-14 18:31:38,334 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796298027_0001_1, queueSize=0
2016-01-14 18:31:38,334 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:38,334 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298027_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796298027_0001_1, startTime=1452796298189, finishTime=1452796298333, timeTaken=144, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1036, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=6, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1177550848, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=9, MERGE_PHASE_TIME=11, FIRST_EVENT_RECEIVED=7, LAST_EVENT_RECEIVED=7, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592596573, Process_Duration=8, Process_End_Time=2905592596581, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:38,334 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796298027_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:38,334 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796298027_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:38,334 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298027_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:38,334 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796298027_0001_00_000002
2016-01-14 18:31:38 Completed Dag: dag_1452796298027_0001_1
2016-01-14 18:31:38,335 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796298027_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:38,335 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796298027_0001_1_post
2016-01-14 18:31:38,335 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,335 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:38,343 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/0_ARGS_SWAP, applicationId=application_1452796298027_0001
2016-01-14 18:31:38,345 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:38,345 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:38,345 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:38,345 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:38,345 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:38,345 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:38,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:38,346 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:38,346 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:38,346 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:38,346 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:38,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:38,347 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:38,351 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/EE39D84A369647AFB77C7FDFC9664C10/FCAA0AC8857849CE9B9D98F05877C9DF/.tez/application_1452796298027_0001
2016-01-14 18:31:38,351 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.343
2016-01-14 18:31:38,418 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//0_ARGS_SWAP
2016-01-14 18:31:38,440 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:38,440 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:38,472 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 18:31:38,476 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 18:31:38,476 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:38,477 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:38,477 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:38,492 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:38,496 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1
2016-01-14 18:31:38,555 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:38,555 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:38,555 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1"]
2016-01-14 18:31:38,555 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:38,556 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:38,556 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:38,559 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/ALL_'field'_-1
2016-01-14 18:31:38,559 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:38,578 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:38,583 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:38,584 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:38,584 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:38,593 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722/.tez/application_1452796298583_0001 doesn't exist and is created
2016-01-14 18:31:38,606 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:38,606 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:38,607 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722/.tez/application_1452796298583_0001
2016-01-14 18:31:38,608 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796298583_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:38,619 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:38,620 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:38,621 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:38,621 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:38,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722/.tez/application_1452796298583_0001/localmode-log-dir
2016-01-14 18:31:38,622 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722/.tez/application_1452796298583_0001/localmode-log-dir/history.txt.appattempt_1452796298583_0001_000000, maxErrors=10
2016-01-14 18:31:38,622 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:38,622 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796298583_0001_000000, appSubmitTime=1452796298608, launchTime=1452796298608
2016-01-14 18:31:38,630 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:38,631 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:49211
2016-01-14 18:31:38,632 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:38,632 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:38,633 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:38,633 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:38,635 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796298583_0001_000000, startTime=1452796298635
2016-01-14 18:31:38,635 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:38,635 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:38,706 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:38,707 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:38,707 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1452796298583_0001, dagName=[5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-01-14 18:31:38,721 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-01-14 18:31:38,723 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796298583_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722/.tez/application_1452796298583_0001/localmode-log-dir/dag_1452796298583_0001_1.dot
2016-01-14 18:31:38,723 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722/.tez/application_1452796298583_0001/localmode-log-dir/dag_1452796298583_0001_1-tez-dag.pb.txt
2016-01-14 18:31:38,730 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796298583_0001_1
2016-01-14 18:31:38,730 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,730 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-01-14 18:31:38 Running Dag: dag_1452796298583_0001_1
2016-01-14 18:31:38,738 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796298583_0001_1, submitTime=1452796298722
2016-01-14 18:31:38,738 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:38,739 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A82085E3DBD1451A8ACB3D7F5D31813E
2016-01-14 18:31:38,739 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:38,739 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796298583_0001_1, initTime=1452796298738
2016-01-14 18:31:38,739 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:38,739 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796298583_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:38,740 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:DAG_STARTED]: dagID=dag_1452796298583_0001_1, startTime=1452796298740
2016-01-14 18:31:38,740 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:38,740 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:38,740 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796298583_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:38,740 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AF2211114D6046D1B6CF169705D027D1 : {C69FC0E2F04441F9ADC7F5EAE47A52BA={InputName=C69FC0E2F04441F9ADC7F5EAE47A52BA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:38,740 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C69FC0E2F04441F9ADC7F5EAE47A52BA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:38,740 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1]
2016-01-14 18:31:38,740 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1]
2016-01-14 18:31:38,740 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1]
2016-01-14 18:31:38,740 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1]. Starting root input initializers: 1
2016-01-14 18:31:38,741 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:38,741 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E]
2016-01-14 18:31:38,743 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C69FC0E2F04441F9ADC7F5EAE47A52BA on vertex vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1]
2016-01-14 18:31:38,743 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1452796298583_0001, dagName=[5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722] every/(1/1) ...lyplatform/ALL_'field'_-1
2016-01-14 18:31:38,744 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:38,744 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E]
2016-01-14 18:31:38,744 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E]
2016-01-14 18:31:38,744 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E]
2016-01-14 18:31:38,745 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=5798A2221E8C4F3D9FFC8C5EC1257219, vertexId=vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:38,745 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=5798A2221E8C4F3D9FFC8C5EC1257219, vertexId=vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E]
2016-01-14 18:31:38,746 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:38,746 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C69FC0E2F04441F9ADC7F5EAE47A52BA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:38,746 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=5798A2221E8C4F3D9FFC8C5EC1257219, vertexName=A82085E3DBD1451A8ACB3D7F5D31813E, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:38,747 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=5798A2221E8C4F3D9FFC8C5EC1257219, vertexId=vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E]
2016-01-14 18:31:38,747 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:38,748 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:38,748 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:38,748 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A82085E3DBD1451A8ACB3D7F5D31813E, vertexId=vertex_1452796298583_0001_1_01, initRequestedTime=1452796298741, initedTime=1452796298748, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:38,748 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:38,749 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:38,749 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C69FC0E2F04441F9ADC7F5EAE47A52BA on vertex vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1]
2016-01-14 18:31:38,750 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:38,750 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C69FC0E2F04441F9ADC7F5EAE47A52BA=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:38,750 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1] parallelism set to 1
2016-01-14 18:31:38,750 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:38,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796298583_0001_1_00, eventCount=1
2016-01-14 18:31:38,751 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:38,751 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AF2211114D6046D1B6CF169705D027D1, vertexId=vertex_1452796298583_0001_1_00, initRequestedTime=1452796298740, initedTime=1452796298751, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:38,751 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1] with distanceFromRoot: 0
2016-01-14 18:31:38,752 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:38,752 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AF2211114D6046D1B6CF169705D027D1
2016-01-14 18:31:38,752 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1]
2016-01-14 18:31:38,752 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:38,752 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796298583_0001_1_00, startRequestedTime=1452796298752, startedTime=1452796298752
2016-01-14 18:31:38,753 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:38,753 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:38,753 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:TASK_STARTED]: vertexName=AF2211114D6046D1B6CF169705D027D1, taskId=task_1452796298583_0001_1_00_000000, scheduledTime=1452796298753, launchTime=1452796298753
2016-01-14 18:31:38,753 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298583_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:38,754 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796298583_0001_1_00 for vertex: vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E] numStartedSources: 1 numSources: 1
2016-01-14 18:31:38,754 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E] with distanceFromRoot: 1
2016-01-14 18:31:38,754 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1]
2016-01-14 18:31:38,754 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AF2211114D6046D1B6CF169705D027D1 in vertex: A82085E3DBD1451A8ACB3D7F5D31813E
2016-01-14 18:31:38,754 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A82085E3DBD1451A8ACB3D7F5D31813E with 1 source tasks and 1 pending tasks
2016-01-14 18:31:38,755 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:38,755 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796298583_0001_1_01, startRequestedTime=1452796298754, startedTime=1452796298754
2016-01-14 18:31:38,755 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:38,755 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: AF2211114D6046D1B6CF169705D027D1, VertexParallelism: 1, TaskAttemptID:attempt_1452796298583_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C69FC0E2F04441F9ADC7F5EAE47A52BA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A82085E3DBD1451A8ACB3D7F5D31813E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:38,755 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298583_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:38,756 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:38,756 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:38,756 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:38,757 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:38,757 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796298583_0001_00_000001
2016-01-14 18:31:38,757 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:38,758 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796298583_0001_00_000001
2016-01-14 18:31:38,758 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:38,759 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:38,759 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:38,759 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796298583_0001_00_000001, launchTime=1452796298759
2016-01-14 18:31:38,760 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796298583_0001_00_000001
2016-01-14 18:31:38,760 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796298583_0001_1_00_000000_0] to container: [container_1452796298583_0001_00_000001]
2016-01-14 18:31:38,760 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:38,761 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796298583_0001_1_00_000000_0] started. Is using containerId: [container_1452796298583_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:38,761 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AF2211114D6046D1B6CF169705D027D1, taskAttemptId=attempt_1452796298583_0001_1_00_000000_0, startTime=1452796298761, containerId=container_1452796298583_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796298583_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:38,761 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298583_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:38,761 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298583_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:38,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:38,762 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796298583_0001_00_000001 given task: attempt_1452796298583_0001_1_00_000000_0
2016-01-14 18:31:38,762 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796298583_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796298583_0001_1_00_000000_0
2016-01-14 18:31:38 Starting to run new task attempt: attempt_1452796298583_0001_1_00_000000_0
2016-01-14 18:31:38,762 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796298583_0001_1_00_000000_0
2016-01-14 18:31:38,762 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796298583_0001, with dag id: dag_1452796298583_0001_1
2016-01-14 18:31:38,763 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,763 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:38,764 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:38,764 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:38,764 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:38,764 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:38,764 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: AF2211114D6046D1B6CF169705D027D1, VertexParallelism: 1, TaskAttemptID:attempt_1452796298583_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C69FC0E2F04441F9ADC7F5EAE47A52BA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A82085E3DBD1451A8ACB3D7F5D31813E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:38,764 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:38,765 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796298583_0001_1_00_000000_0
2016-01-14 18:31:38,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:38,765 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,766 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,767 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796298583_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:38,767 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C69FC0E2F04441F9ADC7F5EAE47A52BA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:38,767 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:38,767 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C69FC0E2F04441F9ADC7F5EAE47A52BA
2016-01-14 18:31:38,768 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C69FC0E2F04441F9ADC7F5EAE47A52BA
2016-01-14 18:31:38,769 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:38,770 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:38,770 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C69FC0E2F04441F9ADC7F5EAE47A52BA
2016-01-14 18:31:38,770 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C69FC0E2F04441F9ADC7F5EAE47A52BA
2016-01-14 18:31:38,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A82085E3DBD1451A8ACB3D7F5D31813E, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:38,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:38,772 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A82085E3DBD1451A8ACB3D7F5D31813E
2016-01-14 18:31:38,774 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:38,775 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A82085E3DBD1451A8ACB3D7F5D31813E
2016-01-14 18:31:38,775 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A82085E3DBD1451A8ACB3D7F5D31813E
2016-01-14 18:31:38,776 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AF2211114D6046D1B6CF169705D027D1, ordinal: 0
2016-01-14 18:31:38,776 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AF2211114D6046D1B6CF169705D027D1, mem on start (mb), free: 203, total: 561, max: 672
2016-01-14 18:31:38,776 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,777 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:38,777 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:38,777 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:38,777 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:38,778 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:38,778 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:38,779 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:38,779 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:38,779 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:38,780 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C69FC0E2F04441F9ADC7F5EAE47A52BA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:38,780 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A82085E3DBD1451A8ACB3D7F5D31813E, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:38,781 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:38,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C69FC0E2F04441F9ADC7F5EAE47A52BA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:38,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:38,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:38,782 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C69FC0E2F04441F9ADC7F5EAE47A52BA
2016-01-14 18:31:38,783 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:38,783 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C69FC0E2F04441F9ADC7F5EAE47A52BA
2016-01-14 18:31:38,783 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:38,783 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796298583_0001_1_00_000000_0
2016-01-14 18:31:38,784 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:38,784 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@13135b00
2016-01-14 18:31:38,785 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@13135b00
2016-01-14 18:31:38,785 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:38,785 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:38,786 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:38,787 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C69FC0E2F04441F9ADC7F5EAE47A52BA
2016-01-14 18:31:38,787 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: AE2C3EAA88EF4EB58F6BF78C89A7B995
2016-01-14 18:31:38,787 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] AE2C3EAA88EF4EB58F6BF78C89A7B995
2016-01-14 18:31:38,787 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@79bb0fed; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6812cf0a; comparator=cascading.tuple.hadoop.util.TupleComparator@2ad6cee7; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:38,788 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:38,788 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:38,788 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:38,811 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:38,811 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:38,811 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:38,811 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:38,811 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:38,812 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:38,812 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:38,813 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AF2211114D6046D1B6CF169705D027D1, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:38,816 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796298583_0001_1_00_000000_0
2016-01-14 18:31:38,816 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AF2211114D6046D1B6CF169705D027D1, mem on close (mb), free: 319, total: 562, max: 672
2016-01-14 18:31:38,816 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:38,817 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:38,817 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:38,817 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:38,818 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:38 Completed running task attempt: attempt_1452796298583_0001_1_00_000000_0
2016-01-14 18:31:38,820 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796298583_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:38,820 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796298813, Process_Duration=3, Process_End_Time=1452796298816, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:38,820 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796298583_0001_1_00, eventCount=1
2016-01-14 18:31:38,821 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:38,821 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,821 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796298583_0001_00_000001
2016-01-14 18:31:38,821 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AF2211114D6046D1B6CF169705D027D1, taskAttemptId=attempt_1452796298583_0001_1_00_000000_0, startTime=1452796298761, finishTime=1452796298821, timeTaken=60, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298813, Process_Duration=3, Process_End_Time=1452796298816, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:38,821 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298583_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:38,821 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:38,821 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796298583_0001_1_00_000000_0
2016-01-14 18:31:38,822 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:38,822 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:TASK_FINISHED]: vertexName=AF2211114D6046D1B6CF169705D027D1, taskId=task_1452796298583_0001_1_00_000000, startTime=1452796298761, finishTime=1452796298821, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1452796298583_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298813, Process_Duration=3, Process_End_Time=1452796298816, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:38,822 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298583_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:38,822 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1] : 1
2016-01-14 18:31:38,822 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1]
2016-01-14 18:31:38,822 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:38,823 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:38,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:VERTEX_FINISHED]: vertexName=AF2211114D6046D1B6CF169705D027D1, vertexId=vertex_1452796298583_0001_1_00, initRequestedTime=1452796298740, initedTime=1452796298751, startRequestedTime=1452796298752, startedTime=1452796298752, finishTime=1452796298822, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298813, Process_Duration=3, Process_End_Time=1452796298816, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796298761, firstTasksToStart=[ task_1452796298583_0001_1_00_000000 ], lastTaskFinishTime=1452796298821, lastTasksToFinish=[ task_1452796298583_0001_1_00_000000 ], minTaskDuration=60, maxTaskDuration=60, avgTaskDuration=60.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796298583_0001_1_00_000000 ], longestDurationTasks=[ task_1452796298583_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:38,824 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:38,824 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E] attempt: attempt_1452796298583_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:38,824 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A82085E3DBD1451A8ACB3D7F5D31813E
2016-01-14 18:31:38,824 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E]
2016-01-14 18:31:38,824 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:38,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796298583_0001_00_000001, stoppedTime=1452796298824, exitStatus=0
2016-01-14 18:31:38,825 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:38,825 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:38,825 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:38,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:TASK_STARTED]: vertexName=A82085E3DBD1451A8ACB3D7F5D31813E, taskId=task_1452796298583_0001_1_01_000000, scheduledTime=1452796298825, launchTime=1452796298825
2016-01-14 18:31:38,825 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298583_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:38,825 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E]
2016-01-14 18:31:38,825 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796298583_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:38,825 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: A82085E3DBD1451A8ACB3D7F5D31813E, VertexParallelism: 1, TaskAttemptID:attempt_1452796298583_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AF2211114D6046D1B6CF169705D027D1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=5798A2221E8C4F3D9FFC8C5EC1257219, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:38,825 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298583_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:38,826 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:38,826 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:38,826 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:38,826 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796298583_0001_00_000002
2016-01-14 18:31:38,826 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:38,828 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796298583_0001_00_000002
2016-01-14 18:31:38,828 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:38,829 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:38,829 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:38,829 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796298583_0001_00_000002, launchTime=1452796298829
2016-01-14 18:31:38,830 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796298583_0001_00_000002
2016-01-14 18:31:38,830 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796298583_0001_1_01_000000_0] to container: [container_1452796298583_0001_00_000002]
2016-01-14 18:31:38,830 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:38,830 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796298583_0001_00_000002 given task: attempt_1452796298583_0001_1_01_000000_0
2016-01-14 18:31:38,830 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796298583_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796298583_0001_1_01_000000_0
2016-01-14 18:31:38,831 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796298583_0001_1_01_000000_0] started. Is using containerId: [container_1452796298583_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:38 Starting to run new task attempt: attempt_1452796298583_0001_1_01_000000_0
2016-01-14 18:31:38,831 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A82085E3DBD1451A8ACB3D7F5D31813E, taskAttemptId=attempt_1452796298583_0001_1_01_000000_0, startTime=1452796298830, containerId=container_1452796298583_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796298583_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:38,831 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:38,831 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796298583_0001_1_01_000000_0
2016-01-14 18:31:38,831 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298583_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:38,831 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298583_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:38,831 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,831 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:38,832 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:38,832 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:38,832 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:38,832 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:38,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722] every/(1/1) ...lyplatform/ALL_'field'_-1, VertexName: A82085E3DBD1451A8ACB3D7F5D31813E, VertexParallelism: 1, TaskAttemptID:attempt_1452796298583_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AF2211114D6046D1B6CF169705D027D1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=5798A2221E8C4F3D9FFC8C5EC1257219, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:38,833 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:38,832 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796298583_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:38,833 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796298583_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:38,833 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796298583_0001_00_000001, Exiting
2016-01-14 18:31:38,834 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796298583_0001_00_000001
2016-01-14 18:31:38,834 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796298583_0001_00_000001
2016-01-14 18:31:38,834 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796298583_0001_1_01_000000_0
2016-01-14 18:31:38,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:38,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,834 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796298583_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:38,835 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796298583_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:38,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=5798A2221E8C4F3D9FFC8C5EC1257219, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:38,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AF2211114D6046D1B6CF169705D027D1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:38,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:38,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:38,835 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796298583_0001_00_000001 completed successfully
2016-01-14 18:31:38,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AF2211114D6046D1B6CF169705D027D1
2016-01-14 18:31:38,836 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:38,838 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:38,838 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 5798A2221E8C4F3D9FFC8C5EC1257219
2016-01-14 18:31:38,838 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:38,839 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:38,840 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: AF2211114D6046D1B6CF169705D027D1
2016-01-14 18:31:38,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AF2211114D6046D1B6CF169705D027D1
2016-01-14 18:31:38,842 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A82085E3DBD1451A8ACB3D7F5D31813E, ordinal: 1
2016-01-14 18:31:38,843 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A82085E3DBD1451A8ACB3D7F5D31813E, mem on start (mb), free: 313, total: 562, max: 672
2016-01-14 18:31:38,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:38,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:38,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:38,843 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:38,843 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 5798A2221E8C4F3D9FFC8C5EC1257219
2016-01-14 18:31:38,851 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:38,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 5798A2221E8C4F3D9FFC8C5EC1257219
2016-01-14 18:31:38,851 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:38,852 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:38,852 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:38,852 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:38,852 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:38,852 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:38,852 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:38,852 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AF2211114D6046D1B6CF169705D027D1, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:38,852 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 5798A2221E8C4F3D9FFC8C5EC1257219, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:38,852 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:38,852 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AF2211114D6046D1B6CF169705D027D1 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:38,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:38,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:38,853 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AF2211114D6046D1B6CF169705D027D1
2016-01-14 18:31:38,854 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:38,854 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:38,854 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:38,854 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:38,854 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AF2211114D6046D1B6CF169705D027D1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:38,854 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:38,854 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:38,854 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:38,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AF2211114D6046D1B6CF169705D027D1
2016-01-14 18:31:38,855 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:38,857 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796298583_0001_1_01_000000_0
2016-01-14 18:31:38,857 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796298583_0001_1_00_000000_0_10231, runDuration: 46507, ]
2016-01-14 18:31:38,858 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796298583_0001_1_00_000000_0_10231] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796298858, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:38,859 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AF2211114D6046D1B6CF169705D027D1
2016-01-14 18:31:38,859 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:38,859 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AF2211114D6046D1B6CF169705D027D1] #39 in 2ms
2016-01-14 18:31:38,860 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AF2211114D6046D1B6CF169705D027D1] #39, status:false, isInterrupted:false
2016-01-14 18:31:38,860 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:38,860 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:38,860 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:38,860 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:38,860 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:38,861 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:38,861 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AF2211114D6046D1B6CF169705D027D1
2016-01-14 18:31:38,861 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:38,861 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: AE2C3EAA88EF4EB58F6BF78C89A7B995
2016-01-14 18:31:38,861 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1"], id: 5798A2221E8C4F3D9FFC8C5EC1257219
2016-01-14 18:31:38,861 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_-1"]
2016-01-14 18:31:38,861 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] AE2C3EAA88EF4EB58F6BF78C89A7B995, for 1 inputs
2016-01-14 18:31:38,862 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A82085E3DBD1451A8ACB3D7F5D31813E, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:38,862 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2ffa9fd2; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6f12ac99
2016-01-14 18:31:38,866 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:38,866 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:38,866 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796298583_0001_1_01_000000_0
2016-01-14 18:31:38,867 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796298583_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:38,867 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796298583_0001_1_01_000000_0
2016-01-14 18:31:38,867 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A82085E3DBD1451A8ACB3D7F5D31813E, mem on close (mb), free: 309, total: 562, max: 672
2016-01-14 18:31:38,867 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AF2211114D6046D1B6CF169705D027D1
2016-01-14 18:31:38,868 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:38,868 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:38,868 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:38,868 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:38,868 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:38 Completed running task attempt: attempt_1452796298583_0001_1_01_000000_0
2016-01-14 18:31:38,869 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796298583_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:38,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796298862, Process_Duration=5, Process_End_Time=1452796298867, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:38,870 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:38,870 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A82085E3DBD1451A8ACB3D7F5D31813E, taskAttemptId=attempt_1452796298583_0001_1_01_000000_0, startTime=1452796298830, finishTime=1452796298870, timeTaken=40, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298862, Process_Duration=5, Process_End_Time=1452796298867, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:38,870 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796298583_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:38,870 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796298583_0001_1_01_000000_0
2016-01-14 18:31:38,870 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:38,870 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:38,870 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:38,871 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:TASK_FINISHED]: vertexName=A82085E3DBD1451A8ACB3D7F5D31813E, taskId=task_1452796298583_0001_1_01_000000, startTime=1452796298830, finishTime=1452796298870, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1452796298583_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298862, Process_Duration=5, Process_End_Time=1452796298867, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:38,871 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796298583_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:38,871 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E] : 1
2016-01-14 18:31:38,871 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E]
2016-01-14 18:31:38,871 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796298583_0001_00_000002
2016-01-14 18:31:38,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:38,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:VERTEX_FINISHED]: vertexName=A82085E3DBD1451A8ACB3D7F5D31813E, vertexId=vertex_1452796298583_0001_1_01, initRequestedTime=1452796298741, initedTime=1452796298748, startRequestedTime=1452796298754, startedTime=1452796298754, finishTime=1452796298871, timeTaken=117, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589299712, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796298862, Process_Duration=5, Process_End_Time=1452796298867, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796298830, firstTasksToStart=[ task_1452796298583_0001_1_01_000000 ], lastTaskFinishTime=1452796298870, lastTasksToFinish=[ task_1452796298583_0001_1_01_000000 ], minTaskDuration=40, maxTaskDuration=40, avgTaskDuration=40.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796298583_0001_1_01_000000 ], longestDurationTasks=[ task_1452796298583_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:38,872 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:38,872 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:38,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796298583_0001_00_000002, stoppedTime=1452796298872, exitStatus=0
2016-01-14 18:31:38,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:38,872 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:38,872 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:38,872 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796298583_0001_1
2016-01-14 18:31:38,873 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796298583_0001_1
2016-01-14 18:31:38,873 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796298583_0001_1_00 [AF2211114D6046D1B6CF169705D027D1]
2016-01-14 18:31:38,873 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 5798A2221E8C4F3D9FFC8C5EC1257219 for vertex: vertex_1452796298583_0001_1_01 [A82085E3DBD1451A8ACB3D7F5D31813E]
2016-01-14 18:31:38,880 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796298583_0001_1, queueSize=0
2016-01-14 18:31:38,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:38,882 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796298583_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796298583_0001_1, startTime=1452796298740, finishTime=1452796298879, timeTaken=139, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1178599424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592597675, Process_Duration=8, Process_End_Time=2905592597683, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:38,882 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796298583_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:38,882 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796298583_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:38,882 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796298583_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:38,883 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796298583_0001_00_000002: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:38,884 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796298583_0001_00_000002, Exiting
2016-01-14 18:31:38,884 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796298583_0001_00_000002
2016-01-14 18:31:38,884 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796298583_0001_00_000002
2016-01-14 18:31:38,884 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796298583_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:38,884 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796298583_0001_00_000002 completed successfully
2016-01-14 18:31:38,884 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796298583_0001_00_000002
2016-01-14 18:31:38 Completed Dag: dag_1452796298583_0001_1
2016-01-14 18:31:38,885 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796298583_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:38,885 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796298583_0001_1_post
2016-01-14 18:31:38,885 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:38,885 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:38,886 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796298583_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:38,891 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_'field'_-1, applicationId=application_1452796298583_0001
2016-01-14 18:31:38,897 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:38,897 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:38,897 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:38,897 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:38,897 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:38,897 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:38,898 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:38,898 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:38,898 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:38,898 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:38,898 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:38,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:38,899 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:38,904 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5C238B20AF114968A5014E43AC414BE0/C4CDAFB442F646FC95A33AC8B2854722/.tez/application_1452796298583_0001
2016-01-14 18:31:38,904 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.349
2016-01-14 18:31:38,972 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_-1
2016-01-14 18:31:38,996 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:38,996 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:39,028 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 18:31:39,034 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 18:31:39,035 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:39,035 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:39,035 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:39,052 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:39,056 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'
2016-01-14 18:31:39,105 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:39,105 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:39,105 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'"]
2016-01-14 18:31:39,105 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:39,105 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:39,106 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:39,107 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...tform/ALL_'field'_'field'
2016-01-14 18:31:39,107 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:39,121 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:39,125 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:39,125 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:39,126 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:39,134 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497/.tez/application_1452796299125_0001 doesn't exist and is created
2016-01-14 18:31:39,149 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:39,149 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:39,149 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497/.tez/application_1452796299125_0001
2016-01-14 18:31:39,150 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796299125_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:39,160 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:39,161 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:39,162 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:39,162 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:39,162 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497/.tez/application_1452796299125_0001/localmode-log-dir
2016-01-14 18:31:39,162 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497/.tez/application_1452796299125_0001/localmode-log-dir/history.txt.appattempt_1452796299125_0001_000000, maxErrors=10
2016-01-14 18:31:39,162 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:39,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796299125_0001_000000, appSubmitTime=1452796299150, launchTime=1452796299150
2016-01-14 18:31:39,165 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:39,166 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:39,166 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:39,168 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:58981
2016-01-14 18:31:39,169 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:39,172 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:39,174 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796299125_0001_000000, startTime=1452796299174
2016-01-14 18:31:39,174 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:39,174 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:39,249 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:39,249 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:39,249 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1452796299125_0001, dagName=[1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497] every/(1/1) ...tform/ALL_'field'_'field'
2016-01-14 18:31:39,266 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497] every/(1/1) ...tform/ALL_'field'_'field'
2016-01-14 18:31:39,267 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796299125_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497/.tez/application_1452796299125_0001/localmode-log-dir/dag_1452796299125_0001_1.dot
2016-01-14 18:31:39,268 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497/.tez/application_1452796299125_0001/localmode-log-dir/dag_1452796299125_0001_1-tez-dag.pb.txt
2016-01-14 18:31:39,274 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796299125_0001_1
2016-01-14 18:31:39,274 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,274 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497] every/(1/1) ...tform/ALL_'field'_'field'
2016-01-14 18:31:39 Running Dag: dag_1452796299125_0001_1
2016-01-14 18:31:39,283 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796299125_0001_1, submitTime=1452796299266
2016-01-14 18:31:39,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:39,284 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C6C4FAB894674BB888D84CE8AF7E583B
2016-01-14 18:31:39,284 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:39,284 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796299125_0001_1, initTime=1452796299283
2016-01-14 18:31:39,284 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796299125_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:39,284 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:39,285 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:DAG_STARTED]: dagID=dag_1452796299125_0001_1, startTime=1452796299285
2016-01-14 18:31:39,285 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:39,285 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796299125_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:39,285 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:39,285 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: CB844CCD128D4FAAB6C0B4B255FC16B7 : {6698DE9A850242669C312FDA81C5221F={InputName=6698DE9A850242669C312FDA81C5221F}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:39,285 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 6698DE9A850242669C312FDA81C5221F, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:39,285 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7]
2016-01-14 18:31:39,285 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7]
2016-01-14 18:31:39,285 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7]
2016-01-14 18:31:39,286 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7]. Starting root input initializers: 1
2016-01-14 18:31:39,286 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:39,286 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 6698DE9A850242669C312FDA81C5221F on vertex vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7]
2016-01-14 18:31:39,286 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B]
2016-01-14 18:31:39,287 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:39,287 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 6698DE9A850242669C312FDA81C5221F asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:39,288 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1452796299125_0001, dagName=[1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497] every/(1/1) ...tform/ALL_'field'_'field'
2016-01-14 18:31:39,289 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:39,289 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B]
2016-01-14 18:31:39,290 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B]
2016-01-14 18:31:39,290 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B]
2016-01-14 18:31:39,290 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=BB10677AA5D441C990E7F387BE6BC489, vertexId=vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:39,290 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=BB10677AA5D441C990E7F387BE6BC489, vertexId=vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B]
2016-01-14 18:31:39,291 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:39,291 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:39,292 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=BB10677AA5D441C990E7F387BE6BC489, vertexName=C6C4FAB894674BB888D84CE8AF7E583B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:39,292 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=BB10677AA5D441C990E7F387BE6BC489, vertexId=vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B]
2016-01-14 18:31:39,293 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:39,293 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:39,293 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C6C4FAB894674BB888D84CE8AF7E583B, vertexId=vertex_1452796299125_0001_1_01, initRequestedTime=1452796299286, initedTime=1452796299293, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:39,293 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:39,294 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 6698DE9A850242669C312FDA81C5221F on vertex vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7]
2016-01-14 18:31:39,294 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:39,294 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {6698DE9A850242669C312FDA81C5221F=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:39,295 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7] parallelism set to 1
2016-01-14 18:31:39,295 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:39,295 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796299125_0001_1_00, eventCount=1
2016-01-14 18:31:39,296 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CB844CCD128D4FAAB6C0B4B255FC16B7, vertexId=vertex_1452796299125_0001_1_00, initRequestedTime=1452796299285, initedTime=1452796299296, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:39,296 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7] with distanceFromRoot: 0
2016-01-14 18:31:39,296 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:39,296 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:39,297 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in CB844CCD128D4FAAB6C0B4B255FC16B7
2016-01-14 18:31:39,297 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7]
2016-01-14 18:31:39,297 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796299125_0001_1_00, startRequestedTime=1452796299297, startedTime=1452796299297
2016-01-14 18:31:39,297 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:39,298 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:39,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:39,298 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:TASK_STARTED]: vertexName=CB844CCD128D4FAAB6C0B4B255FC16B7, taskId=task_1452796299125_0001_1_00_000000, scheduledTime=1452796299298, launchTime=1452796299298
2016-01-14 18:31:39,299 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299125_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:39,299 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796299125_0001_1_00 for vertex: vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B] numStartedSources: 1 numSources: 1
2016-01-14 18:31:39,299 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B] with distanceFromRoot: 1
2016-01-14 18:31:39,299 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7]
2016-01-14 18:31:39,300 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: CB844CCD128D4FAAB6C0B4B255FC16B7 in vertex: C6C4FAB894674BB888D84CE8AF7E583B
2016-01-14 18:31:39,301 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C6C4FAB894674BB888D84CE8AF7E583B with 1 source tasks and 1 pending tasks
2016-01-14 18:31:39,301 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796299125_0001_1_01, startRequestedTime=1452796299300, startedTime=1452796299300
2016-01-14 18:31:39,301 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:39,301 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:39,302 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497] every/(1/1) ...tform/ALL_'field'_'field', VertexName: CB844CCD128D4FAAB6C0B4B255FC16B7, VertexParallelism: 1, TaskAttemptID:attempt_1452796299125_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6698DE9A850242669C312FDA81C5221F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C6C4FAB894674BB888D84CE8AF7E583B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:39,302 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299125_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:39,302 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:39,303 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:39,303 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:39,303 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:39,304 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796299125_0001_00_000001
2016-01-14 18:31:39,304 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:39,305 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796299125_0001, with dag id: dag_1452796299125_0001_1
2016-01-14 18:31:39,306 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796299125_0001_00_000001
2016-01-14 18:31:39,306 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:39,307 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:39,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796299125_0001_00_000001, launchTime=1452796299307
2016-01-14 18:31:39,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:39,308 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796299125_0001_00_000001
2016-01-14 18:31:39,309 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796299125_0001_1_00_000000_0] to container: [container_1452796299125_0001_00_000001]
2016-01-14 18:31:39,309 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:39,309 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796299125_0001_1_00_000000_0] started. Is using containerId: [container_1452796299125_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:39,309 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:39,309 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CB844CCD128D4FAAB6C0B4B255FC16B7, taskAttemptId=attempt_1452796299125_0001_1_00_000000_0, startTime=1452796299309, containerId=container_1452796299125_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796299125_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:39,310 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299125_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:39,310 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299125_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:39,310 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796299125_0001_00_000001 given task: attempt_1452796299125_0001_1_00_000000_0
2016-01-14 18:31:39,310 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796299125_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796299125_0001_1_00_000000_0
2016-01-14 18:31:39 Starting to run new task attempt: attempt_1452796299125_0001_1_00_000000_0
2016-01-14 18:31:39,310 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796299125_0001_1_00_000000_0
2016-01-14 18:31:39,310 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,310 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:39,310 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:39,311 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:39,311 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:39,311 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:39,311 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497] every/(1/1) ...tform/ALL_'field'_'field', VertexName: CB844CCD128D4FAAB6C0B4B255FC16B7, VertexParallelism: 1, TaskAttemptID:attempt_1452796299125_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=6698DE9A850242669C312FDA81C5221F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C6C4FAB894674BB888D84CE8AF7E583B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:39,311 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:39,312 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796299125_0001_1_00_000000_0
2016-01-14 18:31:39,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:39,312 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,312 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796299125_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:39,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C6C4FAB894674BB888D84CE8AF7E583B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:39,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=6698DE9A850242669C312FDA81C5221F, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:39,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:39,313 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:39,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C6C4FAB894674BB888D84CE8AF7E583B
2016-01-14 18:31:39,314 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 6698DE9A850242669C312FDA81C5221F
2016-01-14 18:31:39,314 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 6698DE9A850242669C312FDA81C5221F
2016-01-14 18:31:39,316 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:39,316 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:39,317 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 6698DE9A850242669C312FDA81C5221F
2016-01-14 18:31:39,317 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 6698DE9A850242669C312FDA81C5221F
2016-01-14 18:31:39,318 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:39,319 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C6C4FAB894674BB888D84CE8AF7E583B
2016-01-14 18:31:39,319 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C6C4FAB894674BB888D84CE8AF7E583B
2016-01-14 18:31:39,320 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CB844CCD128D4FAAB6C0B4B255FC16B7, ordinal: 0
2016-01-14 18:31:39,320 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CB844CCD128D4FAAB6C0B4B255FC16B7, mem on start (mb), free: 201, total: 562, max: 672
2016-01-14 18:31:39,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:39,321 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:39,322 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:39,322 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:39,322 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:39,323 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:39,323 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:39,324 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:39,324 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:39,324 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 6698DE9A850242669C312FDA81C5221F, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:39,325 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C6C4FAB894674BB888D84CE8AF7E583B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:39,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:39,325 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 6698DE9A850242669C312FDA81C5221F being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:39,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:39,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:39,326 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 6698DE9A850242669C312FDA81C5221F
2016-01-14 18:31:39,326 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:39,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 6698DE9A850242669C312FDA81C5221F
2016-01-14 18:31:39,327 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:39,327 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796299125_0001_1_00_000000_0
2016-01-14 18:31:39,329 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:39,330 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@35f8647f
2016-01-14 18:31:39,331 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:39,331 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 6698DE9A850242669C312FDA81C5221F
2016-01-14 18:31:39,331 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: A3B73133891141A1BBF3FD21076C7209
2016-01-14 18:31:39,332 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@35f8647f
2016-01-14 18:31:39,332 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:39,332 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:39,332 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] A3B73133891141A1BBF3FD21076C7209
2016-01-14 18:31:39,332 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@445839cc; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7c3aa321; comparator=cascading.tuple.hadoop.util.TupleComparator@6dac0b3e; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:39,332 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:39,333 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:39,333 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:39,350 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:39,350 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:39,357 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:39,357 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:39,357 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:39,357 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:39,357 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:39,358 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:39,358 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:39,358 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CB844CCD128D4FAAB6C0B4B255FC16B7, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:39,362 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796299125_0001_1_00_000000_0
2016-01-14 18:31:39,362 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CB844CCD128D4FAAB6C0B4B255FC16B7, mem on close (mb), free: 317, total: 563, max: 672
2016-01-14 18:31:39,362 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:39,362 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:39,362 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:39,362 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:39,363 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:39 Completed running task attempt: attempt_1452796299125_0001_1_00_000000_0
2016-01-14 18:31:39,366 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796299125_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:39,366 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796299358, Process_Duration=3, Process_End_Time=1452796299361, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:39,366 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796299125_0001_1_00, eventCount=1
2016-01-14 18:31:39,366 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:39,366 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,367 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796299125_0001_00_000001
2016-01-14 18:31:39,367 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CB844CCD128D4FAAB6C0B4B255FC16B7, taskAttemptId=attempt_1452796299125_0001_1_00_000000_0, startTime=1452796299309, finishTime=1452796299366, timeTaken=57, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299358, Process_Duration=3, Process_End_Time=1452796299361, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:39,367 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299125_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:39,367 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:39,367 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:39,367 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796299125_0001_1_00_000000_0
2016-01-14 18:31:39,369 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:39,369 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:TASK_FINISHED]: vertexName=CB844CCD128D4FAAB6C0B4B255FC16B7, taskId=task_1452796299125_0001_1_00_000000, startTime=1452796299309, finishTime=1452796299369, timeTaken=60, status=SUCCEEDED, successfulAttemptID=attempt_1452796299125_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299358, Process_Duration=3, Process_End_Time=1452796299361, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:39,370 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299125_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:39,370 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7] : 1
2016-01-14 18:31:39,370 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7]
2016-01-14 18:31:39,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:39,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:VERTEX_FINISHED]: vertexName=CB844CCD128D4FAAB6C0B4B255FC16B7, vertexId=vertex_1452796299125_0001_1_00, initRequestedTime=1452796299285, initedTime=1452796299296, startRequestedTime=1452796299297, startedTime=1452796299297, finishTime=1452796299370, timeTaken=73, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299358, Process_Duration=3, Process_End_Time=1452796299361, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796299309, firstTasksToStart=[ task_1452796299125_0001_1_00_000000 ], lastTaskFinishTime=1452796299366, lastTasksToFinish=[ task_1452796299125_0001_1_00_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796299125_0001_1_00_000000 ], longestDurationTasks=[ task_1452796299125_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:39,371 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:39,371 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:39,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796299125_0001_00_000001, stoppedTime=1452796299371, exitStatus=0
2016-01-14 18:31:39,371 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:39,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:39,371 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B] attempt: attempt_1452796299125_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:39,371 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C6C4FAB894674BB888D84CE8AF7E583B
2016-01-14 18:31:39,372 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B]
2016-01-14 18:31:39,372 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:39,372 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796299125_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:39,372 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:TASK_STARTED]: vertexName=C6C4FAB894674BB888D84CE8AF7E583B, taskId=task_1452796299125_0001_1_01_000000, scheduledTime=1452796299372, launchTime=1452796299372
2016-01-14 18:31:39,373 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299125_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:39,372 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:39,373 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:39,373 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B]
2016-01-14 18:31:39,373 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497] every/(1/1) ...tform/ALL_'field'_'field', VertexName: C6C4FAB894674BB888D84CE8AF7E583B, VertexParallelism: 1, TaskAttemptID:attempt_1452796299125_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CB844CCD128D4FAAB6C0B4B255FC16B7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BB10677AA5D441C990E7F387BE6BC489, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:39,373 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299125_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:39,374 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:39,374 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796299125_0001_00_000002
2016-01-14 18:31:39,374 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:39,376 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796299125_0001_00_000002
2016-01-14 18:31:39,376 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:39,377 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:39,377 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796299125_0001_00_000002, launchTime=1452796299377
2016-01-14 18:31:39,377 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:39,378 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796299125_0001_00_000002
2016-01-14 18:31:39,378 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796299125_0001_1_01_000000_0] to container: [container_1452796299125_0001_00_000002]
2016-01-14 18:31:39,378 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:39,378 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796299125_0001_00_000002 given task: attempt_1452796299125_0001_1_01_000000_0
2016-01-14 18:31:39,378 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796299125_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796299125_0001_1_01_000000_0
2016-01-14 18:31:39,379 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796299125_0001_1_01_000000_0] started. Is using containerId: [container_1452796299125_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:39 Starting to run new task attempt: attempt_1452796299125_0001_1_01_000000_0
2016-01-14 18:31:39,379 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796299125_0001_1_01_000000_0
2016-01-14 18:31:39,379 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,379 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C6C4FAB894674BB888D84CE8AF7E583B, taskAttemptId=attempt_1452796299125_0001_1_01_000000_0, startTime=1452796299378, containerId=container_1452796299125_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796299125_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:39,379 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:39,379 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:39,379 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:39,380 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:39,380 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:39,380 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497] every/(1/1) ...tform/ALL_'field'_'field', VertexName: C6C4FAB894674BB888D84CE8AF7E583B, VertexParallelism: 1, TaskAttemptID:attempt_1452796299125_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CB844CCD128D4FAAB6C0B4B255FC16B7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BB10677AA5D441C990E7F387BE6BC489, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:39,380 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:39,380 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796299125_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:39,380 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796299125_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:39,380 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796299125_0001_00_000001, Exiting
2016-01-14 18:31:39,380 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796299125_0001_00_000001
2016-01-14 18:31:39,380 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796299125_0001_00_000001
2016-01-14 18:31:39,379 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:39,379 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299125_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:39,381 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299125_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:39,381 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796299125_0001_1_01_000000_0
2016-01-14 18:31:39,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:39,381 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,382 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796299125_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:39,382 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796299125_0001_00_000001 completed successfully
2016-01-14 18:31:39,382 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:39,382 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796299125_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:39,382 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,384 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BB10677AA5D441C990E7F387BE6BC489, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:39,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:39,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BB10677AA5D441C990E7F387BE6BC489
2016-01-14 18:31:39,385 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:39,385 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CB844CCD128D4FAAB6C0B4B255FC16B7, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:39,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:39,386 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CB844CCD128D4FAAB6C0B4B255FC16B7
2016-01-14 18:31:39,387 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:39,387 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:39,387 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: CB844CCD128D4FAAB6C0B4B255FC16B7
2016-01-14 18:31:39,387 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CB844CCD128D4FAAB6C0B4B255FC16B7
2016-01-14 18:31:39,388 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:39,388 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: BB10677AA5D441C990E7F387BE6BC489
2016-01-14 18:31:39,390 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C6C4FAB894674BB888D84CE8AF7E583B, ordinal: 1
2016-01-14 18:31:39,390 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C6C4FAB894674BB888D84CE8AF7E583B, mem on start (mb), free: 311, total: 563, max: 672
2016-01-14 18:31:39,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:39,390 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:39,395 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:39,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BB10677AA5D441C990E7F387BE6BC489
2016-01-14 18:31:39,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:39,395 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:39,395 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:39,396 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:39,396 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:39,396 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:39,396 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:39,396 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CB844CCD128D4FAAB6C0B4B255FC16B7, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:39,397 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BB10677AA5D441C990E7F387BE6BC489, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:39,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:39,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CB844CCD128D4FAAB6C0B4B255FC16B7
2016-01-14 18:31:39,399 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:39,399 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:39,399 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: CB844CCD128D4FAAB6C0B4B255FC16B7 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:39,400 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:39,400 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:39,400 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:39,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CB844CCD128D4FAAB6C0B4B255FC16B7 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:39,400 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CB844CCD128D4FAAB6C0B4B255FC16B7
2016-01-14 18:31:39,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:39,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:39,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:39,402 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796299125_0001_1_01_000000_0
2016-01-14 18:31:39,402 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796299125_0001_1_00_000000_0_10236, runDuration: 50466, ]
2016-01-14 18:31:39,403 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796299125_0001_1_00_000000_0_10236] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796299403, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:39,403 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : CB844CCD128D4FAAB6C0B4B255FC16B7
2016-01-14 18:31:39,403 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:39,403 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [CB844CCD128D4FAAB6C0B4B255FC16B7] #40 in 0ms
2016-01-14 18:31:39,404 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [CB844CCD128D4FAAB6C0B4B255FC16B7] #40, status:false, isInterrupted:false
2016-01-14 18:31:39,406 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:39,406 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: A3B73133891141A1BBF3FD21076C7209
2016-01-14 18:31:39,407 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'"], id: BB10677AA5D441C990E7F387BE6BC489
2016-01-14 18:31:39,406 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:39,407 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_'field'"]
2016-01-14 18:31:39,407 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] A3B73133891141A1BBF3FD21076C7209, for 1 inputs
2016-01-14 18:31:39,407 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:39,407 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:39,408 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:39,408 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:39,408 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : CB844CCD128D4FAAB6C0B4B255FC16B7
2016-01-14 18:31:39,408 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:39,408 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C6C4FAB894674BB888D84CE8AF7E583B, all 1 inputs ready in: 00:00:00.001
2016-01-14 18:31:39,409 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@772d4925; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3e1d5943
2016-01-14 18:31:39,413 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:39,414 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:39,414 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796299125_0001_1_01_000000_0
2016-01-14 18:31:39,414 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796299125_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:39,415 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796299125_0001_1_01_000000_0
2016-01-14 18:31:39,415 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C6C4FAB894674BB888D84CE8AF7E583B, mem on close (mb), free: 308, total: 563, max: 672
2016-01-14 18:31:39,415 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: CB844CCD128D4FAAB6C0B4B255FC16B7
2016-01-14 18:31:39,415 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:39,415 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:39,416 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:39,416 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:39,416 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:39 Completed running task attempt: attempt_1452796299125_0001_1_01_000000_0
2016-01-14 18:31:39,416 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796299125_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:39,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:39,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C6C4FAB894674BB888D84CE8AF7E583B, taskAttemptId=attempt_1452796299125_0001_1_01_000000_0, startTime=1452796299378, finishTime=1452796299417, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299409, Process_Duration=6, Process_End_Time=1452796299415, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:39,417 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299125_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:39,417 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796299125_0001_1_01_000000_0
2016-01-14 18:31:39,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:TASK_FINISHED]: vertexName=C6C4FAB894674BB888D84CE8AF7E583B, taskId=task_1452796299125_0001_1_01_000000, startTime=1452796299378, finishTime=1452796299417, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1452796299125_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299409, Process_Duration=6, Process_End_Time=1452796299415, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:39,418 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299125_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:39,418 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B] : 1
2016-01-14 18:31:39,418 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B]
2016-01-14 18:31:39,418 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:39,418 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:39,418 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:VERTEX_FINISHED]: vertexName=C6C4FAB894674BB888D84CE8AF7E583B, vertexId=vertex_1452796299125_0001_1_01, initRequestedTime=1452796299286, initedTime=1452796299293, startRequestedTime=1452796299300, startedTime=1452796299300, finishTime=1452796299418, timeTaken=118, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299409, Process_Duration=6, Process_End_Time=1452796299415, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796299378, firstTasksToStart=[ task_1452796299125_0001_1_01_000000 ], lastTaskFinishTime=1452796299417, lastTasksToFinish=[ task_1452796299125_0001_1_01_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796299125_0001_1_01_000000 ], longestDurationTasks=[ task_1452796299125_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:39,418 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:39,418 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:39,419 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:39,419 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796299125_0001_00_000002, stoppedTime=1452796299419, exitStatus=0
2016-01-14 18:31:39,419 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:39,419 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:39,419 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796299125_0001_1
2016-01-14 18:31:39,419 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796299125_0001_1
2016-01-14 18:31:39,419 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796299125_0001_1_00 [CB844CCD128D4FAAB6C0B4B255FC16B7]
2016-01-14 18:31:39,419 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: BB10677AA5D441C990E7F387BE6BC489 for vertex: vertex_1452796299125_0001_1_01 [C6C4FAB894674BB888D84CE8AF7E583B]
2016-01-14 18:31:39,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:39,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=652, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796299409, Process_Duration=6, Process_End_Time=1452796299415, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:39,421 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:39,421 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,421 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796299125_0001_00_000002
2016-01-14 18:31:39,421 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796299125_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:39,421 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796299125_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:39,421 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796299125_0001_00_000002, Exiting
2016-01-14 18:31:39,421 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796299125_0001_00_000002
2016-01-14 18:31:39,421 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796299125_0001_00_000002
2016-01-14 18:31:39,422 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796299125_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:39,422 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796299125_0001_00_000002 completed successfully
2016-01-14 18:31:39,427 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796299125_0001_1, queueSize=0
2016-01-14 18:31:39,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:39,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299125_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796299125_0001_1, startTime=1452796299285, finishTime=1452796299426, timeTaken=141, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=985, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1180696576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592598767, Process_Duration=9, Process_End_Time=2905592598776, Read_Duration=4, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:39,427 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796299125_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:39,427 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796299125_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:39,427 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299125_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:39,427 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796299125_0001_00_000002
2016-01-14 18:31:39 Completed Dag: dag_1452796299125_0001_1
2016-01-14 18:31:39,428 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796299125_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:39,428 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796299125_0001_1_post
2016-01-14 18:31:39,428 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,428 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:39,435 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_'field'_'field', applicationId=application_1452796299125_0001
2016-01-14 18:31:39,436 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:39,436 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:39,436 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:39,436 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:39,436 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:39,437 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:39,437 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:39,437 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:39,437 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:39,438 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:39,438 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:39,438 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:39,439 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:39,444 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/1DFBFDD959894627ABFAC3F5A04718E5/C97922E02CBC465EA9A8D3217418C497/.tez/application_1452796299125_0001
2016-01-14 18:31:39,444 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.339
2016-01-14 18:31:39,509 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_'field'
2016-01-14 18:31:39,530 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:39,530 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:39,564 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 18:31:39,565 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 18:31:39,566 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:39,566 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:39,566 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:39,579 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:39,584 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS
2016-01-14 18:31:39,627 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:39,627 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:39,627 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS"]
2016-01-14 18:31:39,627 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:39,628 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:39,628 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:39,629 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...tform/ALL_'field'_RESULTS
2016-01-14 18:31:39,629 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:39,641 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:39,645 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:39,645 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:39,646 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:39,655 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6/.tez/application_1452796299645_0001 doesn't exist and is created
2016-01-14 18:31:39,668 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:39,668 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:39,668 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6/.tez/application_1452796299645_0001
2016-01-14 18:31:39,670 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796299645_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:39,680 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:39,681 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:39,682 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:39,682 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:39,682 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6/.tez/application_1452796299645_0001/localmode-log-dir
2016-01-14 18:31:39,683 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6/.tez/application_1452796299645_0001/localmode-log-dir/history.txt.appattempt_1452796299645_0001_000000, maxErrors=10
2016-01-14 18:31:39,683 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:39,683 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796299645_0001_000000, appSubmitTime=1452796299670, launchTime=1452796299670
2016-01-14 18:31:39,685 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:39,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:39,686 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:39,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:39,688 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:43262
2016-01-14 18:31:39,690 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:39,691 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796299645_0001_000000, startTime=1452796299691
2016-01-14 18:31:39,691 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:39,692 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:39,768 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:39,768 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:39,768 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1452796299645_0001, dagName=[0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-01-14 18:31:39,785 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-01-14 18:31:39,786 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796299645_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6/.tez/application_1452796299645_0001/localmode-log-dir/dag_1452796299645_0001_1.dot
2016-01-14 18:31:39,786 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6/.tez/application_1452796299645_0001/localmode-log-dir/dag_1452796299645_0001_1-tez-dag.pb.txt
2016-01-14 18:31:39,793 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796299645_0001_1
2016-01-14 18:31:39,793 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,793 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-01-14 18:31:39 Running Dag: dag_1452796299645_0001_1
2016-01-14 18:31:39,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796299645_0001_1, submitTime=1452796299785
2016-01-14 18:31:39,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:39,803 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex C083B6ADF98143698895D984A3EBD798
2016-01-14 18:31:39,803 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:39,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796299645_0001_1, initTime=1452796299803
2016-01-14 18:31:39,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:39,804 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796299645_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:39,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:DAG_STARTED]: dagID=dag_1452796299645_0001_1, startTime=1452796299804
2016-01-14 18:31:39,804 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:39,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:39,804 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796299645_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:39,804 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: FA3747AA9C11433791A68437C50E1A86 : {2D9981B4E8A448DB9FD56FF45558E167={InputName=2D9981B4E8A448DB9FD56FF45558E167}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:39,804 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 2D9981B4E8A448DB9FD56FF45558E167, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:39,805 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86]
2016-01-14 18:31:39,805 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86]
2016-01-14 18:31:39,805 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86]
2016-01-14 18:31:39,805 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86]. Starting root input initializers: 1
2016-01-14 18:31:39,805 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:39,805 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 2D9981B4E8A448DB9FD56FF45558E167 on vertex vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86]
2016-01-14 18:31:39,805 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798]
2016-01-14 18:31:39,807 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1452796299645_0001, dagName=[0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6] every/(1/1) ...tform/ALL_'field'_RESULTS
2016-01-14 18:31:39,807 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:39,807 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 2D9981B4E8A448DB9FD56FF45558E167 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:39,807 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:39,808 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:39,809 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:39,809 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798]
2016-01-14 18:31:39,809 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:39,809 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798]
2016-01-14 18:31:39,809 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 2D9981B4E8A448DB9FD56FF45558E167 on vertex vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86]
2016-01-14 18:31:39,809 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798]
2016-01-14 18:31:39,809 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=2C30906F14584FEDB618FC09062CF0C5, vertexId=vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:39,810 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=2C30906F14584FEDB618FC09062CF0C5, vertexId=vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798]
2016-01-14 18:31:39,811 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=2C30906F14584FEDB618FC09062CF0C5, vertexName=C083B6ADF98143698895D984A3EBD798, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:39,811 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=2C30906F14584FEDB618FC09062CF0C5, vertexId=vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798]
2016-01-14 18:31:39,812 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:39,812 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C083B6ADF98143698895D984A3EBD798, vertexId=vertex_1452796299645_0001_1_01, initRequestedTime=1452796299805, initedTime=1452796299812, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:39,812 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:39,812 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:39,812 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {2D9981B4E8A448DB9FD56FF45558E167=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:39,813 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86] parallelism set to 1
2016-01-14 18:31:39,813 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:39,813 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796299645_0001_1_00, eventCount=1
2016-01-14 18:31:39,813 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:39,814 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FA3747AA9C11433791A68437C50E1A86, vertexId=vertex_1452796299645_0001_1_00, initRequestedTime=1452796299804, initedTime=1452796299813, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:39,814 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86] with distanceFromRoot: 0
2016-01-14 18:31:39,814 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:39,814 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in FA3747AA9C11433791A68437C50E1A86
2016-01-14 18:31:39,814 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86]
2016-01-14 18:31:39,814 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:39,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796299645_0001_1_00, startRequestedTime=1452796299814, startedTime=1452796299814
2016-01-14 18:31:39,815 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:39,815 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:39,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:TASK_STARTED]: vertexName=FA3747AA9C11433791A68437C50E1A86, taskId=task_1452796299645_0001_1_00_000000, scheduledTime=1452796299815, launchTime=1452796299815
2016-01-14 18:31:39,816 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299645_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:39,816 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796299645_0001_1_00 for vertex: vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798] numStartedSources: 1 numSources: 1
2016-01-14 18:31:39,816 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798] with distanceFromRoot: 1
2016-01-14 18:31:39,816 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86]
2016-01-14 18:31:39,816 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: FA3747AA9C11433791A68437C50E1A86 in vertex: C083B6ADF98143698895D984A3EBD798
2016-01-14 18:31:39,816 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: C083B6ADF98143698895D984A3EBD798 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:39,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:39,817 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796299645_0001_1_01, startRequestedTime=1452796299816, startedTime=1452796299816
2016-01-14 18:31:39,817 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:39,817 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: FA3747AA9C11433791A68437C50E1A86, VertexParallelism: 1, TaskAttemptID:attempt_1452796299645_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2D9981B4E8A448DB9FD56FF45558E167, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C083B6ADF98143698895D984A3EBD798, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:39,817 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299645_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:39,818 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:39,818 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:39,819 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:39,819 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:39,819 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796299645_0001_00_000001
2016-01-14 18:31:39,819 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:39,821 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796299645_0001_00_000001
2016-01-14 18:31:39,821 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:39,821 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:39,821 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796299645_0001_00_000001, launchTime=1452796299821
2016-01-14 18:31:39,821 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:39,822 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796299645_0001_00_000001
2016-01-14 18:31:39,823 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796299645_0001_1_00_000000_0] to container: [container_1452796299645_0001_00_000001]
2016-01-14 18:31:39,823 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:39,823 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796299645_0001_00_000001 given task: attempt_1452796299645_0001_1_00_000000_0
2016-01-14 18:31:39,823 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796299645_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796299645_0001_1_00_000000_0
2016-01-14 18:31:39,823 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796299645_0001_1_00_000000_0] started. Is using containerId: [container_1452796299645_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:39 Starting to run new task attempt: attempt_1452796299645_0001_1_00_000000_0
2016-01-14 18:31:39,824 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796299645_0001_1_00_000000_0
2016-01-14 18:31:39,824 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,824 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:39,824 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:39,824 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:39,824 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:39,824 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:39,824 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: FA3747AA9C11433791A68437C50E1A86, VertexParallelism: 1, TaskAttemptID:attempt_1452796299645_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=2D9981B4E8A448DB9FD56FF45558E167, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=C083B6ADF98143698895D984A3EBD798, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:39,824 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:39,825 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:39,825 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FA3747AA9C11433791A68437C50E1A86, taskAttemptId=attempt_1452796299645_0001_1_00_000000_0, startTime=1452796299823, containerId=container_1452796299645_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796299645_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:39,825 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299645_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:39,825 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299645_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:39,827 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796299645_0001, with dag id: dag_1452796299645_0001_1
2016-01-14 18:31:39,827 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796299645_0001_1_00_000000_0
2016-01-14 18:31:39,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:39,827 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,828 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796299645_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:39,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=2D9981B4E8A448DB9FD56FF45558E167, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:39,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C083B6ADF98143698895D984A3EBD798, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:39,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:39,830 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C083B6ADF98143698895D984A3EBD798
2016-01-14 18:31:39,832 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:39,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:39,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 2D9981B4E8A448DB9FD56FF45558E167
2016-01-14 18:31:39,833 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 2D9981B4E8A448DB9FD56FF45558E167
2016-01-14 18:31:39,834 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:39,834 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: C083B6ADF98143698895D984A3EBD798
2016-01-14 18:31:39,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C083B6ADF98143698895D984A3EBD798
2016-01-14 18:31:39,835 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:39,835 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 2D9981B4E8A448DB9FD56FF45558E167
2016-01-14 18:31:39,835 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 2D9981B4E8A448DB9FD56FF45558E167
2016-01-14 18:31:39,837 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FA3747AA9C11433791A68437C50E1A86, ordinal: 0
2016-01-14 18:31:39,837 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA3747AA9C11433791A68437C50E1A86, mem on start (mb), free: 201, total: 563, max: 672
2016-01-14 18:31:39,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:39,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:39,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:39,837 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:39,837 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:39,838 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:39,838 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:39,838 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:39,838 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:39,838 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 2D9981B4E8A448DB9FD56FF45558E167, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:39,838 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C083B6ADF98143698895D984A3EBD798, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:39,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:39,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 2D9981B4E8A448DB9FD56FF45558E167 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:39,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:39,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:39,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 2D9981B4E8A448DB9FD56FF45558E167
2016-01-14 18:31:39,839 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:39,839 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 2D9981B4E8A448DB9FD56FF45558E167
2016-01-14 18:31:39,840 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:39,840 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796299645_0001_1_00_000000_0
2016-01-14 18:31:39,840 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:39,841 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@7a4402f
2016-01-14 18:31:39,841 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@7a4402f
2016-01-14 18:31:39,841 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:39,842 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:39,843 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:39,843 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 2D9981B4E8A448DB9FD56FF45558E167
2016-01-14 18:31:39,844 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: F403608EB4C1492DB4744F9D24FC1232
2016-01-14 18:31:39,844 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] F403608EB4C1492DB4744F9D24FC1232
2016-01-14 18:31:39,844 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@63934d10; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5ba88882; comparator=cascading.tuple.hadoop.util.TupleComparator@159a3dd1; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:39,844 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:39,845 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:39,845 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:39,859 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:39,859 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:39,867 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:39,867 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:39,867 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:39,867 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:39,867 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:39,868 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:39,868 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:39,869 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FA3747AA9C11433791A68437C50E1A86, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:39,870 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796299645_0001_1_00_000000_0
2016-01-14 18:31:39,871 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA3747AA9C11433791A68437C50E1A86, mem on close (mb), free: 310, total: 558, max: 672
2016-01-14 18:31:39,871 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:39,871 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:39,871 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:39,871 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:39,872 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:39 Completed running task attempt: attempt_1452796299645_0001_1_00_000000_0
2016-01-14 18:31:39,874 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796299645_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:39,875 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=585629696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796299869, Process_Duration=1, Process_End_Time=1452796299870, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:39,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796299645_0001_1_00, eventCount=1
2016-01-14 18:31:39,875 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:39,875 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,875 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796299645_0001_00_000001
2016-01-14 18:31:39,875 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FA3747AA9C11433791A68437C50E1A86, taskAttemptId=attempt_1452796299645_0001_1_00_000000_0, startTime=1452796299823, finishTime=1452796299875, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=585629696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299869, Process_Duration=1, Process_End_Time=1452796299870, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:39,875 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299645_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:39,876 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:39,876 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796299645_0001_1_00_000000_0
2016-01-14 18:31:39,876 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:TASK_FINISHED]: vertexName=FA3747AA9C11433791A68437C50E1A86, taskId=task_1452796299645_0001_1_00_000000, startTime=1452796299823, finishTime=1452796299876, timeTaken=53, status=SUCCEEDED, successfulAttemptID=attempt_1452796299645_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=585629696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299869, Process_Duration=1, Process_End_Time=1452796299870, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:39,876 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299645_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:39,876 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86] : 1
2016-01-14 18:31:39,876 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86]
2016-01-14 18:31:39,877 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:VERTEX_FINISHED]: vertexName=FA3747AA9C11433791A68437C50E1A86, vertexId=vertex_1452796299645_0001_1_00, initRequestedTime=1452796299804, initedTime=1452796299813, startRequestedTime=1452796299814, startedTime=1452796299814, finishTime=1452796299876, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=585629696, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299869, Process_Duration=1, Process_End_Time=1452796299870, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796299823, firstTasksToStart=[ task_1452796299645_0001_1_00_000000 ], lastTaskFinishTime=1452796299875, lastTasksToFinish=[ task_1452796299645_0001_1_00_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796299645_0001_1_00_000000 ], longestDurationTasks=[ task_1452796299645_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:39,876 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:39,877 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:39,877 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:39,878 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:39,878 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796299645_0001_00_000001, stoppedTime=1452796299877, exitStatus=0
2016-01-14 18:31:39,878 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:39,878 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:39,878 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798] attempt: attempt_1452796299645_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:39,878 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: C083B6ADF98143698895D984A3EBD798
2016-01-14 18:31:39,878 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798]
2016-01-14 18:31:39,878 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:39,879 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:39,879 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796299645_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:39,879 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:TASK_STARTED]: vertexName=C083B6ADF98143698895D984A3EBD798, taskId=task_1452796299645_0001_1_01_000000, scheduledTime=1452796299879, launchTime=1452796299879
2016-01-14 18:31:39,879 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299645_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:39,880 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:39,880 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798]
2016-01-14 18:31:39,880 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: C083B6ADF98143698895D984A3EBD798, VertexParallelism: 1, TaskAttemptID:attempt_1452796299645_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FA3747AA9C11433791A68437C50E1A86, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2C30906F14584FEDB618FC09062CF0C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:39,881 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299645_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:39,879 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:39,881 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:39,881 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796299645_0001_00_000002
2016-01-14 18:31:39,882 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:39,883 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796299645_0001_00_000002
2016-01-14 18:31:39,883 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:39,883 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:39,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796299645_0001_00_000002, launchTime=1452796299883
2016-01-14 18:31:39,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:39,884 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796299645_0001_00_000002
2016-01-14 18:31:39,885 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796299645_0001_1_01_000000_0] to container: [container_1452796299645_0001_00_000002]
2016-01-14 18:31:39,885 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:39,885 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796299645_0001_1_01_000000_0] started. Is using containerId: [container_1452796299645_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:39,885 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796299645_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:39,886 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C083B6ADF98143698895D984A3EBD798, taskAttemptId=attempt_1452796299645_0001_1_01_000000_0, startTime=1452796299885, containerId=container_1452796299645_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796299645_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:39,886 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299645_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:39,886 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299645_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:39,886 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:39,886 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796299645_0001_00_000002 given task: attempt_1452796299645_0001_1_01_000000_0
2016-01-14 18:31:39,886 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796299645_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796299645_0001_1_01_000000_0
2016-01-14 18:31:39 Starting to run new task attempt: attempt_1452796299645_0001_1_01_000000_0
2016-01-14 18:31:39,887 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796299645_0001_1_01_000000_0
2016-01-14 18:31:39,887 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,887 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:39,887 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:39,887 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:39,887 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:39,887 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:39,887 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6] every/(1/1) ...tform/ALL_'field'_RESULTS, VertexName: C083B6ADF98143698895D984A3EBD798, VertexParallelism: 1, TaskAttemptID:attempt_1452796299645_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=FA3747AA9C11433791A68437C50E1A86, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=2C30906F14584FEDB618FC09062CF0C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:39,888 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:39,888 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796299645_0001_1_01_000000_0
2016-01-14 18:31:39,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:39,888 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,887 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796299645_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:39,888 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796299645_0001_00_000001, Exiting
2016-01-14 18:31:39,888 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796299645_0001_00_000001
2016-01-14 18:31:39,888 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796299645_0001_00_000001
2016-01-14 18:31:39,889 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,889 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796299645_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:39,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=FA3747AA9C11433791A68437C50E1A86, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:39,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:39,890 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: FA3747AA9C11433791A68437C50E1A86
2016-01-14 18:31:39,891 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:39,892 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:39,893 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:39,893 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: FA3747AA9C11433791A68437C50E1A86
2016-01-14 18:31:39,893 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: FA3747AA9C11433791A68437C50E1A86
2016-01-14 18:31:39,894 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C083B6ADF98143698895D984A3EBD798, ordinal: 1
2016-01-14 18:31:39,894 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C083B6ADF98143698895D984A3EBD798, mem on start (mb), free: 305, total: 558, max: 672
2016-01-14 18:31:39,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:39,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:39,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:39,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=2C30906F14584FEDB618FC09062CF0C5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:39,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:39,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 2C30906F14584FEDB618FC09062CF0C5
2016-01-14 18:31:39,895 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:39,895 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 2C30906F14584FEDB618FC09062CF0C5
2016-01-14 18:31:39,896 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796299645_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:39,896 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796299645_0001_00_000001 completed successfully
2016-01-14 18:31:39,896 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:39,905 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:39,905 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 2C30906F14584FEDB618FC09062CF0C5
2016-01-14 18:31:39,907 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:39,907 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:39,907 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:39,907 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:39,907 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:39,907 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:39,907 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:39,907 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, FA3747AA9C11433791A68437C50E1A86, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:39,908 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 2C30906F14584FEDB618FC09062CF0C5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:39,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:39,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: FA3747AA9C11433791A68437C50E1A86 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:39,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:39,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:39,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: FA3747AA9C11433791A68437C50E1A86
2016-01-14 18:31:39,908 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:39,909 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:39,909 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: FA3747AA9C11433791A68437C50E1A86 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:39,909 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:39,909 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:39,909 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:39,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: FA3747AA9C11433791A68437C50E1A86
2016-01-14 18:31:39,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:39,910 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796299645_0001_1_01_000000_0
2016-01-14 18:31:39,911 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796299645_0001_1_00_000000_0_10242, runDuration: 42888, ]
2016-01-14 18:31:39,912 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796299645_0001_1_00_000000_0_10242] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796299912, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:39,912 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : FA3747AA9C11433791A68437C50E1A86
2016-01-14 18:31:39,912 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:39,913 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:39,913 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [FA3747AA9C11433791A68437C50E1A86] #41 in 1ms
2016-01-14 18:31:39,913 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [FA3747AA9C11433791A68437C50E1A86] #41, status:false, isInterrupted:false
2016-01-14 18:31:39,913 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: F403608EB4C1492DB4744F9D24FC1232
2016-01-14 18:31:39,913 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS"], id: 2C30906F14584FEDB618FC09062CF0C5
2016-01-14 18:31:39,913 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_RESULTS"]
2016-01-14 18:31:39,913 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:39,914 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] F403608EB4C1492DB4744F9D24FC1232, for 1 inputs
2016-01-14 18:31:39,914 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:39,914 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:39,914 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:39,914 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:39,914 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : FA3747AA9C11433791A68437C50E1A86
2016-01-14 18:31:39,915 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:39,915 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C083B6ADF98143698895D984A3EBD798, all 1 inputs ready in: 00:00:00.001
2016-01-14 18:31:39,915 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@67fd5d12; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2ae97e14
2016-01-14 18:31:39,917 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:39,918 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:39,918 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796299645_0001_1_01_000000_0
2016-01-14 18:31:39,918 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796299645_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:39,919 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796299645_0001_1_01_000000_0
2016-01-14 18:31:39,919 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C083B6ADF98143698895D984A3EBD798, mem on close (mb), free: 301, total: 558, max: 672
2016-01-14 18:31:39,919 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: FA3747AA9C11433791A68437C50E1A86
2016-01-14 18:31:39,919 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:39,920 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:39,920 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:39,920 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:39,920 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:39 Completed running task attempt: attempt_1452796299645_0001_1_01_000000_0
2016-01-14 18:31:39,921 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796299645_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:39,921 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:39,921 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C083B6ADF98143698895D984A3EBD798, taskAttemptId=attempt_1452796299645_0001_1_01_000000_0, startTime=1452796299885, finishTime=1452796299921, timeTaken=36, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=585629696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299915, Process_Duration=4, Process_End_Time=1452796299919, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:39,922 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796299645_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:39,922 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796299645_0001_1_01_000000_0
2016-01-14 18:31:39,922 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:39,922 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:TASK_FINISHED]: vertexName=C083B6ADF98143698895D984A3EBD798, taskId=task_1452796299645_0001_1_01_000000, startTime=1452796299885, finishTime=1452796299922, timeTaken=37, status=SUCCEEDED, successfulAttemptID=attempt_1452796299645_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=585629696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299915, Process_Duration=4, Process_End_Time=1452796299919, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:39,922 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796299645_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:39,922 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798] : 1
2016-01-14 18:31:39,922 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798]
2016-01-14 18:31:39,923 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:39,923 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:39,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:VERTEX_FINISHED]: vertexName=C083B6ADF98143698895D984A3EBD798, vertexId=vertex_1452796299645_0001_1_01, initRequestedTime=1452796299805, initedTime=1452796299812, startRequestedTime=1452796299816, startedTime=1452796299816, finishTime=1452796299923, timeTaken=107, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=585629696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796299915, Process_Duration=4, Process_End_Time=1452796299919, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796299885, firstTasksToStart=[ task_1452796299645_0001_1_01_000000 ], lastTaskFinishTime=1452796299921, lastTasksToFinish=[ task_1452796299645_0001_1_01_000000 ], minTaskDuration=36, maxTaskDuration=36, avgTaskDuration=36.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796299645_0001_1_01_000000 ], longestDurationTasks=[ task_1452796299645_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:39,923 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:39,924 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:39,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796299645_0001_00_000002, stoppedTime=1452796299924, exitStatus=0
2016-01-14 18:31:39,924 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:39,924 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:39,924 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796299645_0001_1
2016-01-14 18:31:39,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796299645_0001_1
2016-01-14 18:31:39,924 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 2C30906F14584FEDB618FC09062CF0C5 for vertex: vertex_1452796299645_0001_1_01 [C083B6ADF98143698895D984A3EBD798]
2016-01-14 18:31:39,925 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:39,925 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=585629696, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796299915, Process_Duration=4, Process_End_Time=1452796299919, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:39,925 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:39,926 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,926 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796299645_0001_00_000002
2016-01-14 18:31:39,926 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796299645_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:39,926 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796299645_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:39,926 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796299645_0001_00_000002, Exiting
2016-01-14 18:31:39,926 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796299645_0001_00_000002
2016-01-14 18:31:39,927 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796299645_0001_00_000002
2016-01-14 18:31:39,927 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796299645_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:39,927 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796299645_0001_00_000002 completed successfully
2016-01-14 18:31:39,932 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796299645_0001_1_00 [FA3747AA9C11433791A68437C50E1A86]
2016-01-14 18:31:39,933 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796299645_0001_1, queueSize=0
2016-01-14 18:31:39,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:39,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796299645_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796299645_0001_1, startTime=1452796299804, finishTime=1452796299932, timeTaken=128, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=985, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1171259392, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=5, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592599784, Process_Duration=5, Process_End_Time=2905592599789, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:39,933 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796299645_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:39,933 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796299645_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:39,934 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796299645_0001_00_000002
2016-01-14 18:31:39,934 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796299645_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:39 Completed Dag: dag_1452796299645_0001_1
2016-01-14 18:31:39,935 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796299645_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:39,936 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796299645_0001_1_post
2016-01-14 18:31:39,936 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:39,936 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:39,940 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_'field'_RESULTS, applicationId=application_1452796299645_0001
2016-01-14 18:31:39,941 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:39,941 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:39,941 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:39,941 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:39,941 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:39,941 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:39,942 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:39,942 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:39,942 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:39,942 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:39,943 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:39,943 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:39,943 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:39,948 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/0914204CC65845D4BF2892B05382977F/AE3BFE36420240EE849EB90940A78EE6/.tez/application_1452796299645_0001
2016-01-14 18:31:39,948 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.321
2016-01-14 18:31:40,016 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_RESULTS
2016-01-14 18:31:40,038 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:40,039 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:40,071 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 18:31:40,078 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 18:31:40,079 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:40,079 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:40,080 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:40,095 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:40,100 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL
2016-01-14 18:31:40,147 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:40,147 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:40,147 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL"]
2016-01-14 18:31:40,147 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:40,148 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:40,148 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:40,148 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/ALL_'field'_ALL
2016-01-14 18:31:40,148 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:40,160 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:40,164 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:40,164 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:40,164 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:40,173 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE/.tez/application_1452796300164_0001 doesn't exist and is created
2016-01-14 18:31:40,187 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:40,187 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:40,187 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE/.tez/application_1452796300164_0001
2016-01-14 18:31:40,191 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796300164_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:40,201 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:40,202 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:40,202 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:40,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:40,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE/.tez/application_1452796300164_0001/localmode-log-dir
2016-01-14 18:31:40,203 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE/.tez/application_1452796300164_0001/localmode-log-dir/history.txt.appattempt_1452796300164_0001_000000, maxErrors=10
2016-01-14 18:31:40,203 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:40,203 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796300164_0001_000000, appSubmitTime=1452796300191, launchTime=1452796300191
2016-01-14 18:31:40,206 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:40,206 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:40,207 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:40,208 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:40,208 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:41122
2016-01-14 18:31:40,210 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:40,211 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796300164_0001_000000, startTime=1452796300211
2016-01-14 18:31:40,211 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:40,211 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:40,287 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:40,287 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:40,287 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1452796300164_0001, dagName=[5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-01-14 18:31:40,302 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-01-14 18:31:40,304 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796300164_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE/.tez/application_1452796300164_0001/localmode-log-dir/dag_1452796300164_0001_1.dot
2016-01-14 18:31:40,304 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE/.tez/application_1452796300164_0001/localmode-log-dir/dag_1452796300164_0001_1-tez-dag.pb.txt
2016-01-14 18:31:40,310 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796300164_0001_1
2016-01-14 18:31:40,310 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:40,310 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-01-14 18:31:40 Running Dag: dag_1452796300164_0001_1
2016-01-14 18:31:40,319 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796300164_0001_1, submitTime=1452796300303
2016-01-14 18:31:40,319 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:40,320 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex EEFBBCD3950645B2B74319A3F429B1C3
2016-01-14 18:31:40,320 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:40,320 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796300164_0001_1, initTime=1452796300319
2016-01-14 18:31:40,320 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:40,321 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796300164_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:40,321 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:DAG_STARTED]: dagID=dag_1452796300164_0001_1, startTime=1452796300321
2016-01-14 18:31:40,321 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:40,321 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796300164_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:40,321 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B01D5BF8E49B49DF8E155A75B8C52895 : {BCAE2E4A4E284807B51F39586DD1B25C={InputName=BCAE2E4A4E284807B51F39586DD1B25C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:40,321 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: BCAE2E4A4E284807B51F39586DD1B25C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:40,321 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895]
2016-01-14 18:31:40,321 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895]
2016-01-14 18:31:40,322 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895]
2016-01-14 18:31:40,322 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895]. Starting root input initializers: 1
2016-01-14 18:31:40,322 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:40,322 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3]
2016-01-14 18:31:40,321 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:40,322 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: BCAE2E4A4E284807B51F39586DD1B25C on vertex vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895]
2016-01-14 18:31:40,324 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:40,325 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3]
2016-01-14 18:31:40,325 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3]
2016-01-14 18:31:40,325 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3]
2016-01-14 18:31:40,325 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=23883A41CD414EE3BF35BFE3AF828A32, vertexId=vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:40,325 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=23883A41CD414EE3BF35BFE3AF828A32, vertexId=vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3]
2016-01-14 18:31:40,326 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:40,326 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input BCAE2E4A4E284807B51F39586DD1B25C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:40,326 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1452796300164_0001, dagName=[5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE] every/(1/1) ...yplatform/ALL_'field'_ALL
2016-01-14 18:31:40,326 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:40,327 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:40,328 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=23883A41CD414EE3BF35BFE3AF828A32, vertexName=EEFBBCD3950645B2B74319A3F429B1C3, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:40,328 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:40,328 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: BCAE2E4A4E284807B51F39586DD1B25C on vertex vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895]
2016-01-14 18:31:40,328 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=23883A41CD414EE3BF35BFE3AF828A32, vertexId=vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3]
2016-01-14 18:31:40,329 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:40,329 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:VERTEX_INITIALIZED]: vertexName=EEFBBCD3950645B2B74319A3F429B1C3, vertexId=vertex_1452796300164_0001_1_01, initRequestedTime=1452796300322, initedTime=1452796300329, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:40,329 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:40,329 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:40,330 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {BCAE2E4A4E284807B51F39586DD1B25C=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:40,330 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895] parallelism set to 1
2016-01-14 18:31:40,330 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:40,330 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796300164_0001_1_00, eventCount=1
2016-01-14 18:31:40,331 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B01D5BF8E49B49DF8E155A75B8C52895, vertexId=vertex_1452796300164_0001_1_00, initRequestedTime=1452796300321, initedTime=1452796300331, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:40,331 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:40,331 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895] with distanceFromRoot: 0
2016-01-14 18:31:40,331 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:40,331 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B01D5BF8E49B49DF8E155A75B8C52895
2016-01-14 18:31:40,331 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895]
2016-01-14 18:31:40,332 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:40,332 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796300164_0001_1_00, startRequestedTime=1452796300331, startedTime=1452796300331
2016-01-14 18:31:40,332 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:40,333 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:40,333 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:TASK_STARTED]: vertexName=B01D5BF8E49B49DF8E155A75B8C52895, taskId=task_1452796300164_0001_1_00_000000, scheduledTime=1452796300333, launchTime=1452796300333
2016-01-14 18:31:40,333 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300164_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:40,334 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796300164_0001_1_00 for vertex: vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3] numStartedSources: 1 numSources: 1
2016-01-14 18:31:40,334 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3] with distanceFromRoot: 1
2016-01-14 18:31:40,334 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:40,334 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:40,334 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895]
2016-01-14 18:31:40,335 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B01D5BF8E49B49DF8E155A75B8C52895 in vertex: EEFBBCD3950645B2B74319A3F429B1C3
2016-01-14 18:31:40,335 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: EEFBBCD3950645B2B74319A3F429B1C3 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:40,336 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:40,336 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796300164_0001_1_01, startRequestedTime=1452796300335, startedTime=1452796300335
2016-01-14 18:31:40,336 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:40,337 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: B01D5BF8E49B49DF8E155A75B8C52895, VertexParallelism: 1, TaskAttemptID:attempt_1452796300164_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BCAE2E4A4E284807B51F39586DD1B25C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EEFBBCD3950645B2B74319A3F429B1C3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:40,337 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300164_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:40,337 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:40,338 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:40,338 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:40,338 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:40,339 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796300164_0001_00_000001
2016-01-14 18:31:40,339 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:40,341 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796300164_0001_00_000001
2016-01-14 18:31:40,341 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:40,342 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:40,342 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796300164_0001_00_000001, launchTime=1452796300342
2016-01-14 18:31:40,342 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:40,343 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796300164_0001, with dag id: dag_1452796300164_0001_1
2016-01-14 18:31:40,343 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796300164_0001_00_000001
2016-01-14 18:31:40,343 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796300164_0001_1_00_000000_0] to container: [container_1452796300164_0001_00_000001]
2016-01-14 18:31:40,344 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:40,344 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796300164_0001_00_000001 given task: attempt_1452796300164_0001_1_00_000000_0
2016-01-14 18:31:40,344 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796300164_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796300164_0001_1_00_000000_0
2016-01-14 18:31:40 Starting to run new task attempt: attempt_1452796300164_0001_1_00_000000_0
2016-01-14 18:31:40,344 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796300164_0001_1_00_000000_0
2016-01-14 18:31:40,344 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:40,345 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:40,345 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:40,345 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:40,345 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:40,345 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:40,345 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: B01D5BF8E49B49DF8E155A75B8C52895, VertexParallelism: 1, TaskAttemptID:attempt_1452796300164_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=BCAE2E4A4E284807B51F39586DD1B25C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=EEFBBCD3950645B2B74319A3F429B1C3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:40,345 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:40,346 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796300164_0001_1_00_000000_0] started. Is using containerId: [container_1452796300164_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:40,346 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:40,346 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B01D5BF8E49B49DF8E155A75B8C52895, taskAttemptId=attempt_1452796300164_0001_1_00_000000_0, startTime=1452796300344, containerId=container_1452796300164_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796300164_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:40,347 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300164_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:40,347 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300164_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:40,348 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796300164_0001_1_00_000000_0
2016-01-14 18:31:40,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:40,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,348 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796300164_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:40,349 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=BCAE2E4A4E284807B51F39586DD1B25C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:40,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:40,350 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: BCAE2E4A4E284807B51F39586DD1B25C
2016-01-14 18:31:40,350 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: BCAE2E4A4E284807B51F39586DD1B25C
2016-01-14 18:31:40,352 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:40,352 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: BCAE2E4A4E284807B51F39586DD1B25C
2016-01-14 18:31:40,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: BCAE2E4A4E284807B51F39586DD1B25C
2016-01-14 18:31:40,352 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EEFBBCD3950645B2B74319A3F429B1C3, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:40,352 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:40,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:40,353 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EEFBBCD3950645B2B74319A3F429B1C3
2016-01-14 18:31:40,354 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:40,354 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: EEFBBCD3950645B2B74319A3F429B1C3
2016-01-14 18:31:40,354 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EEFBBCD3950645B2B74319A3F429B1C3
2016-01-14 18:31:40,356 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B01D5BF8E49B49DF8E155A75B8C52895, ordinal: 0
2016-01-14 18:31:40,356 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B01D5BF8E49B49DF8E155A75B8C52895, mem on start (mb), free: 194, total: 558, max: 672
2016-01-14 18:31:40,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:40,356 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:40,357 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:40,357 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:40,357 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:40,357 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:40,357 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:40,357 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:40,358 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:40,358 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, BCAE2E4A4E284807B51F39586DD1B25C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:40,358 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EEFBBCD3950645B2B74319A3F429B1C3, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:40,358 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:40,358 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: BCAE2E4A4E284807B51F39586DD1B25C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:40,358 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: BCAE2E4A4E284807B51F39586DD1B25C
2016-01-14 18:31:40,359 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:40,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: BCAE2E4A4E284807B51F39586DD1B25C
2016-01-14 18:31:40,359 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:40,360 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:40,360 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:40,360 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796300164_0001_1_00_000000_0
2016-01-14 18:31:40,362 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:40,363 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: BCAE2E4A4E284807B51F39586DD1B25C
2016-01-14 18:31:40,363 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:40,363 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: D20408F42B3E45588438166B776C24AD
2016-01-14 18:31:40,363 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] D20408F42B3E45588438166B776C24AD
2016-01-14 18:31:40,364 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2280266d; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@b5acfcd; comparator=cascading.tuple.hadoop.util.TupleComparator@7700a18d; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:40,364 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:40,364 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:40,364 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:40,375 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@1316347
2016-01-14 18:31:40,376 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@1316347
2016-01-14 18:31:40,377 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:40,377 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:40,387 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:40,387 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:40,387 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:40,387 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:40,388 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:40,388 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:40,388 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:40,389 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B01D5BF8E49B49DF8E155A75B8C52895, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:40,392 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796300164_0001_1_00_000000_0
2016-01-14 18:31:40,392 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B01D5BF8E49B49DF8E155A75B8C52895, mem on close (mb), free: 311, total: 562, max: 672
2016-01-14 18:31:40,392 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:40,392 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:40,392 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:40,392 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:40,394 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:40 Completed running task attempt: attempt_1452796300164_0001_1_00_000000_0
2016-01-14 18:31:40,397 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796300164_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:40,397 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796300164_0001_1_00, eventCount=1
2016-01-14 18:31:40,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796300389, Process_Duration=2, Process_End_Time=1452796300391, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:40,398 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:40,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B01D5BF8E49B49DF8E155A75B8C52895, taskAttemptId=attempt_1452796300164_0001_1_00_000000_0, startTime=1452796300344, finishTime=1452796300398, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796300389, Process_Duration=2, Process_End_Time=1452796300391, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:40,398 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300164_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:40,398 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796300164_0001_1_00_000000_0
2016-01-14 18:31:40,398 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:40,398 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:40,398 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:TASK_FINISHED]: vertexName=B01D5BF8E49B49DF8E155A75B8C52895, taskId=task_1452796300164_0001_1_00_000000, startTime=1452796300344, finishTime=1452796300398, timeTaken=54, status=SUCCEEDED, successfulAttemptID=attempt_1452796300164_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796300389, Process_Duration=2, Process_End_Time=1452796300391, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:40,399 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796300164_0001_00_000001
2016-01-14 18:31:40,399 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:40,399 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300164_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:40,399 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895] : 1
2016-01-14 18:31:40,399 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895]
2016-01-14 18:31:40,400 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:VERTEX_FINISHED]: vertexName=B01D5BF8E49B49DF8E155A75B8C52895, vertexId=vertex_1452796300164_0001_1_00, initRequestedTime=1452796300321, initedTime=1452796300331, startRequestedTime=1452796300331, startedTime=1452796300331, finishTime=1452796300399, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796300389, Process_Duration=2, Process_End_Time=1452796300391, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796300344, firstTasksToStart=[ task_1452796300164_0001_1_00_000000 ], lastTaskFinishTime=1452796300398, lastTasksToFinish=[ task_1452796300164_0001_1_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796300164_0001_1_00_000000 ], longestDurationTasks=[ task_1452796300164_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:40,399 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:40,400 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:40,400 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:40,400 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:40,400 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796300164_0001_00_000001, stoppedTime=1452796300400, exitStatus=0
2016-01-14 18:31:40,400 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:40,400 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3] attempt: attempt_1452796300164_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:40,401 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: EEFBBCD3950645B2B74319A3F429B1C3
2016-01-14 18:31:40,401 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:40,401 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3]
2016-01-14 18:31:40,401 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:40,401 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:TASK_STARTED]: vertexName=EEFBBCD3950645B2B74319A3F429B1C3, taskId=task_1452796300164_0001_1_01_000000, scheduledTime=1452796300401, launchTime=1452796300401
2016-01-14 18:31:40,401 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300164_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:40,401 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3]
2016-01-14 18:31:40,402 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: EEFBBCD3950645B2B74319A3F429B1C3, VertexParallelism: 1, TaskAttemptID:attempt_1452796300164_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B01D5BF8E49B49DF8E155A75B8C52895, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=23883A41CD414EE3BF35BFE3AF828A32, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:40,402 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300164_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:40,402 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796300164_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:40,402 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:40,402 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:40,402 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:40,403 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796300164_0001_00_000002
2016-01-14 18:31:40,403 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:40,405 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796300164_0001_00_000002
2016-01-14 18:31:40,405 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:40,405 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:40,405 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:40,406 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796300164_0001_00_000002, launchTime=1452796300405
2016-01-14 18:31:40,406 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796300164_0001_00_000002
2016-01-14 18:31:40,407 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796300164_0001_1_01_000000_0] to container: [container_1452796300164_0001_00_000002]
2016-01-14 18:31:40,407 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:40,407 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796300164_0001_1_01_000000_0] started. Is using containerId: [container_1452796300164_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:40,407 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=EEFBBCD3950645B2B74319A3F429B1C3, taskAttemptId=attempt_1452796300164_0001_1_01_000000_0, startTime=1452796300407, containerId=container_1452796300164_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796300164_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:40,407 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300164_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:40,407 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300164_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:40,408 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:40,408 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796300164_0001_00_000002 given task: attempt_1452796300164_0001_1_01_000000_0
2016-01-14 18:31:40,408 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796300164_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796300164_0001_1_01_000000_0
2016-01-14 18:31:40 Starting to run new task attempt: attempt_1452796300164_0001_1_01_000000_0
2016-01-14 18:31:40,408 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796300164_0001_1_01_000000_0
2016-01-14 18:31:40,409 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:40,409 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:40,409 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:40,409 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:40,409 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:40,409 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:40,409 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796300164_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:40,409 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE] every/(1/1) ...yplatform/ALL_'field'_ALL, VertexName: EEFBBCD3950645B2B74319A3F429B1C3, VertexParallelism: 1, TaskAttemptID:attempt_1452796300164_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B01D5BF8E49B49DF8E155A75B8C52895, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=23883A41CD414EE3BF35BFE3AF828A32, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:40,409 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796300164_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:40,409 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:40,410 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796300164_0001_00_000001, Exiting
2016-01-14 18:31:40,410 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796300164_0001_00_000001
2016-01-14 18:31:40,410 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796300164_0001_00_000001
2016-01-14 18:31:40,410 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796300164_0001_1_01_000000_0
2016-01-14 18:31:40,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:40,410 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,411 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796300164_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:40,411 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796300164_0001_00_000001 completed successfully
2016-01-14 18:31:40,411 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B01D5BF8E49B49DF8E155A75B8C52895, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:40,411 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796300164_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:40,411 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:40,411 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:40,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B01D5BF8E49B49DF8E155A75B8C52895
2016-01-14 18:31:40,412 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=23883A41CD414EE3BF35BFE3AF828A32, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:40,413 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:40,413 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 23883A41CD414EE3BF35BFE3AF828A32
2016-01-14 18:31:40,413 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:40,413 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 23883A41CD414EE3BF35BFE3AF828A32
2016-01-14 18:31:40,415 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:40,416 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:40,416 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:40,416 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B01D5BF8E49B49DF8E155A75B8C52895
2016-01-14 18:31:40,416 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B01D5BF8E49B49DF8E155A75B8C52895
2016-01-14 18:31:40,420 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: EEFBBCD3950645B2B74319A3F429B1C3, ordinal: 1
2016-01-14 18:31:40,420 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EEFBBCD3950645B2B74319A3F429B1C3, mem on start (mb), free: 305, total: 562, max: 672
2016-01-14 18:31:40,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:40,420 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:40,421 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:40,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 23883A41CD414EE3BF35BFE3AF828A32
2016-01-14 18:31:40,421 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:40,421 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:40,422 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:40,422 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:40,422 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:40,422 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:40,422 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:40,422 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 23883A41CD414EE3BF35BFE3AF828A32, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:40,422 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B01D5BF8E49B49DF8E155A75B8C52895, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:40,422 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:40,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B01D5BF8E49B49DF8E155A75B8C52895 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:40,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B01D5BF8E49B49DF8E155A75B8C52895
2016-01-14 18:31:40,423 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:40,423 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:40,423 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:40,424 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:40,424 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B01D5BF8E49B49DF8E155A75B8C52895 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:40,424 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:40,424 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:40,424 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:40,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B01D5BF8E49B49DF8E155A75B8C52895
2016-01-14 18:31:40,425 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:40,425 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796300164_0001_1_01_000000_0
2016-01-14 18:31:40,427 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:40,428 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: D20408F42B3E45588438166B776C24AD
2016-01-14 18:31:40,428 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL"], id: 23883A41CD414EE3BF35BFE3AF828A32
2016-01-14 18:31:40,428 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_ALL"]
2016-01-14 18:31:40,428 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] D20408F42B3E45588438166B776C24AD, for 1 inputs
2016-01-14 18:31:40,429 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796300164_0001_1_00_000000_0_10249, runDuration: 42645, ]
2016-01-14 18:31:40,429 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796300164_0001_1_00_000000_0_10249] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796300429, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:40,430 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B01D5BF8E49B49DF8E155A75B8C52895
2016-01-14 18:31:40,430 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:40,430 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B01D5BF8E49B49DF8E155A75B8C52895] #42 in 1ms
2016-01-14 18:31:40,430 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B01D5BF8E49B49DF8E155A75B8C52895] #42, status:false, isInterrupted:false
2016-01-14 18:31:40,431 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:40,431 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:40,431 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:40,431 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:40,432 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:40,432 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: EEFBBCD3950645B2B74319A3F429B1C3, all 1 inputs ready in: 00:00:00.004
2016-01-14 18:31:40,432 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B01D5BF8E49B49DF8E155A75B8C52895
2016-01-14 18:31:40,432 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2f42c6; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@1ec8d542
2016-01-14 18:31:40,433 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:40,436 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:40,436 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:40,436 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796300164_0001_1_01_000000_0
2016-01-14 18:31:40,436 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796300164_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:40,437 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796300164_0001_1_01_000000_0
2016-01-14 18:31:40,437 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: EEFBBCD3950645B2B74319A3F429B1C3, mem on close (mb), free: 302, total: 562, max: 672
2016-01-14 18:31:40,437 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B01D5BF8E49B49DF8E155A75B8C52895
2016-01-14 18:31:40,437 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:40,437 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:40,438 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:40,438 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:40,438 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:40 Completed running task attempt: attempt_1452796300164_0001_1_01_000000_0
2016-01-14 18:31:40,439 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796300164_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:40,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796300432, Process_Duration=5, Process_End_Time=1452796300437, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:40,439 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=EEFBBCD3950645B2B74319A3F429B1C3, taskAttemptId=attempt_1452796300164_0001_1_01_000000_0, startTime=1452796300407, finishTime=1452796300439, timeTaken=32, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796300432, Process_Duration=5, Process_End_Time=1452796300437, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:40,439 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:40,439 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:40,440 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796300164_0001_00_000002
2016-01-14 18:31:40,440 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:40,439 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300164_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:40,440 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:40,440 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796300164_0001_1_01_000000_0
2016-01-14 18:31:40,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:TASK_FINISHED]: vertexName=EEFBBCD3950645B2B74319A3F429B1C3, taskId=task_1452796300164_0001_1_01_000000, startTime=1452796300407, finishTime=1452796300441, timeTaken=34, status=SUCCEEDED, successfulAttemptID=attempt_1452796300164_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796300432, Process_Duration=5, Process_End_Time=1452796300437, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:40,441 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300164_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:40,441 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:40,442 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3] : 1
2016-01-14 18:31:40,442 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3]
2016-01-14 18:31:40,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:40,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:VERTEX_FINISHED]: vertexName=EEFBBCD3950645B2B74319A3F429B1C3, vertexId=vertex_1452796300164_0001_1_01, initRequestedTime=1452796300322, initedTime=1452796300329, startRequestedTime=1452796300335, startedTime=1452796300335, finishTime=1452796300442, timeTaken=107, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796300432, Process_Duration=5, Process_End_Time=1452796300437, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796300407, firstTasksToStart=[ task_1452796300164_0001_1_01_000000 ], lastTaskFinishTime=1452796300439, lastTasksToFinish=[ task_1452796300164_0001_1_01_000000 ], minTaskDuration=32, maxTaskDuration=32, avgTaskDuration=32.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796300164_0001_1_01_000000 ], longestDurationTasks=[ task_1452796300164_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:40,443 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:40,443 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:40,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796300164_0001_00_000002, stoppedTime=1452796300443, exitStatus=0
2016-01-14 18:31:40,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:40,443 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:40,444 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:40,444 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796300164_0001_1
2016-01-14 18:31:40,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796300164_0001_1
2016-01-14 18:31:40,444 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796300164_0001_1_00 [B01D5BF8E49B49DF8E155A75B8C52895]
2016-01-14 18:31:40,444 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 23883A41CD414EE3BF35BFE3AF828A32 for vertex: vertex_1452796300164_0001_1_01 [EEFBBCD3950645B2B74319A3F429B1C3]
2016-01-14 18:31:40,450 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796300164_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:40,450 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796300164_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:40,450 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796300164_0001_00_000002, Exiting
2016-01-14 18:31:40,450 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796300164_0001_00_000002
2016-01-14 18:31:40,450 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796300164_0001_00_000002
2016-01-14 18:31:40,451 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796300164_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:40,451 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796300164_0001_00_000002 completed successfully
2016-01-14 18:31:40,452 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796300164_0001_1, queueSize=0
2016-01-14 18:31:40,452 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:40,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300164_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796300164_0001_1, startTime=1452796300321, finishTime=1452796300450, timeTaken=129, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1091, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1179648000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=7, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=6, LAST_EVENT_RECEIVED=6, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592600821, Process_Duration=7, Process_End_Time=2905592600828, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:40,452 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796300164_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:40,452 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796300164_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:40,452 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796300164_0001_00_000002
2016-01-14 18:31:40,453 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300164_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:40 Completed Dag: dag_1452796300164_0001_1
2016-01-14 18:31:40,453 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796300164_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:40,453 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796300164_0001_1_post
2016-01-14 18:31:40,453 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:40,453 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:40,460 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_'field'_ALL, applicationId=application_1452796300164_0001
2016-01-14 18:31:40,462 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:40,462 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:40,462 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:40,462 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:40,462 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:40,462 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:40,462 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:40,463 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:40,463 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:40,463 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:40,462 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:40,463 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:40,463 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:40,467 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/5FC217E8DC8E483487AE7C0690A4C148/F0FF623A5A3B44709F3ECA3D5C402BFE/.tez/application_1452796300164_0001
2016-01-14 18:31:40,468 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.321
2016-01-14 18:31:40,535 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_ALL
2016-01-14 18:31:40,557 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:40,557 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:40,584 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 18:31:40,585 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-01-14 18:31:40,585 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:40,585 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:40,586 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_REPLACE
2016-01-14 18:31:40,608 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:40,608 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:40,637 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-01-14 18:31:40,643 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 18:31:40,643 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:40,643 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:40,644 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:40,658 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:40,662 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP
2016-01-14 18:31:40,706 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:40,706 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:40,706 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP"]
2016-01-14 18:31:40,706 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:40,706 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:40,706 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:40,708 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/ALL_'field'_SWAP
2016-01-14 18:31:40,708 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:40,720 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:40,724 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:40,724 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:40,724 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:40,733 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873/.tez/application_1452796300724_0001 doesn't exist and is created
2016-01-14 18:31:40,746 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:40,746 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:40,746 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873/.tez/application_1452796300724_0001
2016-01-14 18:31:40,747 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796300724_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:40,757 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:40,759 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:40,760 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:40,760 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:40,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873/.tez/application_1452796300724_0001/localmode-log-dir
2016-01-14 18:31:40,761 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873/.tez/application_1452796300724_0001/localmode-log-dir/history.txt.appattempt_1452796300724_0001_000000, maxErrors=10
2016-01-14 18:31:40,761 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:40,762 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796300724_0001_000000, appSubmitTime=1452796300747, launchTime=1452796300747
2016-01-14 18:31:40,764 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:40,764 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:40,766 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:40,766 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:40,767 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:37257
2016-01-14 18:31:40,769 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:40,770 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796300724_0001_000000, startTime=1452796300770
2016-01-14 18:31:40,770 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:40,770 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:40,812 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:40,812 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:40,846 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:40,846 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:40,846 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1452796300724_0001, dagName=[30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873] every/(1/1) ...platform/ALL_'field'_SWAP
2016-01-14 18:31:40,862 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873] every/(1/1) ...platform/ALL_'field'_SWAP
2016-01-14 18:31:40,863 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796300724_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873/.tez/application_1452796300724_0001/localmode-log-dir/dag_1452796300724_0001_1.dot
2016-01-14 18:31:40,864 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873/.tez/application_1452796300724_0001/localmode-log-dir/dag_1452796300724_0001_1-tez-dag.pb.txt
2016-01-14 18:31:40,870 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796300724_0001_1
2016-01-14 18:31:40,870 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:40,870 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873] every/(1/1) ...platform/ALL_'field'_SWAP
2016-01-14 18:31:40 Running Dag: dag_1452796300724_0001_1
2016-01-14 18:31:40,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796300724_0001_1, submitTime=1452796300862
2016-01-14 18:31:40,880 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:40,881 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex F39D92C0B67E4D54941368A5AF7BE5BF
2016-01-14 18:31:40,881 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:40,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796300724_0001_1, initTime=1452796300880
2016-01-14 18:31:40,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:40,881 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796300724_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:40,882 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:DAG_STARTED]: dagID=dag_1452796300724_0001_1, startTime=1452796300882
2016-01-14 18:31:40,882 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:40,882 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796300724_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:40,882 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:40,882 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F8925C37892C4368A625E693EDAF478A : {D2693E8F2412454C9D523EC809D0576E={InputName=D2693E8F2412454C9D523EC809D0576E}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:40,882 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D2693E8F2412454C9D523EC809D0576E, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:40,882 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A]
2016-01-14 18:31:40,882 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1452796300724_0001, dagName=[30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873] every/(1/1) ...platform/ALL_'field'_SWAP
2016-01-14 18:31:40,883 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A]
2016-01-14 18:31:40,883 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A]
2016-01-14 18:31:40,883 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A]. Starting root input initializers: 1
2016-01-14 18:31:40,883 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:40,883 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF]
2016-01-14 18:31:40,887 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:40,887 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF]
2016-01-14 18:31:40,887 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF]
2016-01-14 18:31:40,887 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF]
2016-01-14 18:31:40,887 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=B65AC996999B452BB5FB696FAB17A812, vertexId=vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:40,887 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=B65AC996999B452BB5FB696FAB17A812, vertexId=vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF]
2016-01-14 18:31:40,887 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D2693E8F2412454C9D523EC809D0576E on vertex vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A]
2016-01-14 18:31:40,889 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=B65AC996999B452BB5FB696FAB17A812, vertexName=F39D92C0B67E4D54941368A5AF7BE5BF, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:40,889 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=B65AC996999B452BB5FB696FAB17A812, vertexId=vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF]
2016-01-14 18:31:40,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:40,890 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:40,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F39D92C0B67E4D54941368A5AF7BE5BF, vertexId=vertex_1452796300724_0001_1_01, initRequestedTime=1452796300883, initedTime=1452796300890, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:40,890 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:40,891 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D2693E8F2412454C9D523EC809D0576E asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:40,891 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:40,891 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:40,893 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:40,893 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D2693E8F2412454C9D523EC809D0576E on vertex vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A]
2016-01-14 18:31:40,893 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:40,893 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D2693E8F2412454C9D523EC809D0576E=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:40,894 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A] parallelism set to 1
2016-01-14 18:31:40,894 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:40,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796300724_0001_1_00, eventCount=1
2016-01-14 18:31:40,894 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:40,894 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F8925C37892C4368A625E693EDAF478A, vertexId=vertex_1452796300724_0001_1_00, initRequestedTime=1452796300882, initedTime=1452796300894, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:40,895 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A] with distanceFromRoot: 0
2016-01-14 18:31:40,895 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:40,895 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F8925C37892C4368A625E693EDAF478A
2016-01-14 18:31:40,895 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A]
2016-01-14 18:31:40,895 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:40,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796300724_0001_1_00, startRequestedTime=1452796300895, startedTime=1452796300895
2016-01-14 18:31:40,896 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:40,896 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:40,896 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:TASK_STARTED]: vertexName=F8925C37892C4368A625E693EDAF478A, taskId=task_1452796300724_0001_1_00_000000, scheduledTime=1452796300896, launchTime=1452796300896
2016-01-14 18:31:40,897 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300724_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:40,897 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796300724_0001_1_00 for vertex: vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF] numStartedSources: 1 numSources: 1
2016-01-14 18:31:40,897 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF] with distanceFromRoot: 1
2016-01-14 18:31:40,897 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A]
2016-01-14 18:31:40,897 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F8925C37892C4368A625E693EDAF478A in vertex: F39D92C0B67E4D54941368A5AF7BE5BF
2016-01-14 18:31:40,898 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: F39D92C0B67E4D54941368A5AF7BE5BF with 1 source tasks and 1 pending tasks
2016-01-14 18:31:40,898 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:40,907 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796300724_0001_1_01, startRequestedTime=1452796300897, startedTime=1452796300897
2016-01-14 18:31:40,907 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:40,907 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: F8925C37892C4368A625E693EDAF478A, VertexParallelism: 1, TaskAttemptID:attempt_1452796300724_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D2693E8F2412454C9D523EC809D0576E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F39D92C0B67E4D54941368A5AF7BE5BF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:40,908 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300724_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:40,908 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:40,908 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:40,909 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:40,909 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:40,909 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796300724_0001_00_000001
2016-01-14 18:31:40,909 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:40,911 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796300724_0001_00_000001
2016-01-14 18:31:40,911 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:40,913 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:40,913 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:40,913 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796300724_0001_00_000001, launchTime=1452796300913
2016-01-14 18:31:40,914 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796300724_0001_00_000001
2016-01-14 18:31:40,914 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796300724_0001_1_00_000000_0] to container: [container_1452796300724_0001_00_000001]
2016-01-14 18:31:40,914 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:40,915 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796300724_0001_00_000001 given task: attempt_1452796300724_0001_1_00_000000_0
2016-01-14 18:31:40,915 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796300724_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796300724_0001_1_00_000000_0
2016-01-14 18:31:40,915 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796300724_0001, with dag id: dag_1452796300724_0001_1
2016-01-14 18:31:40,915 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796300724_0001_1_00_000000_0] started. Is using containerId: [container_1452796300724_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:40 Starting to run new task attempt: attempt_1452796300724_0001_1_00_000000_0
2016-01-14 18:31:40,915 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796300724_0001_1_00_000000_0
2016-01-14 18:31:40,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:40,916 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:40,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F8925C37892C4368A625E693EDAF478A, taskAttemptId=attempt_1452796300724_0001_1_00_000000_0, startTime=1452796300915, containerId=container_1452796300724_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796300724_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:40,916 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:40,917 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:40,917 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:40,917 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:40,917 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:40,917 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: F8925C37892C4368A625E693EDAF478A, VertexParallelism: 1, TaskAttemptID:attempt_1452796300724_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D2693E8F2412454C9D523EC809D0576E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=F39D92C0B67E4D54941368A5AF7BE5BF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:40,917 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:40,916 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300724_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:40,918 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796300724_0001_1_00_000000_0
2016-01-14 18:31:40,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:40,918 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,918 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300724_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:40,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D2693E8F2412454C9D523EC809D0576E, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:40,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=F39D92C0B67E4D54941368A5AF7BE5BF, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:40,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:40,919 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,919 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796300724_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:40,920 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:40,921 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: F39D92C0B67E4D54941368A5AF7BE5BF
2016-01-14 18:31:40,922 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:40,922 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: F39D92C0B67E4D54941368A5AF7BE5BF
2016-01-14 18:31:40,922 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: F39D92C0B67E4D54941368A5AF7BE5BF
2016-01-14 18:31:40,923 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:40,924 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D2693E8F2412454C9D523EC809D0576E
2016-01-14 18:31:40,926 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D2693E8F2412454C9D523EC809D0576E
2016-01-14 18:31:40,928 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:40,928 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D2693E8F2412454C9D523EC809D0576E
2016-01-14 18:31:40,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D2693E8F2412454C9D523EC809D0576E
2016-01-14 18:31:40,930 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F8925C37892C4368A625E693EDAF478A, ordinal: 0
2016-01-14 18:31:40,930 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8925C37892C4368A625E693EDAF478A, mem on start (mb), free: 404, total: 562, max: 672
2016-01-14 18:31:40,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:40,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:40,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:40,930 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:40,930 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:40,930 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:40,931 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:40,931 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:40,931 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:40,931 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, F39D92C0B67E4D54941368A5AF7BE5BF, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:40,931 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D2693E8F2412454C9D523EC809D0576E, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:40,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:40,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D2693E8F2412454C9D523EC809D0576E being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:40,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:40,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:40,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D2693E8F2412454C9D523EC809D0576E
2016-01-14 18:31:40,932 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:40,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D2693E8F2412454C9D523EC809D0576E
2016-01-14 18:31:40,932 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:40,932 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796300724_0001_1_00_000000_0
2016-01-14 18:31:40,932 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:40,933 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@5938baba
2016-01-14 18:31:40,933 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@5938baba
2016-01-14 18:31:40,934 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:40,934 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:40,935 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:40,936 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D2693E8F2412454C9D523EC809D0576E
2016-01-14 18:31:40,936 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 3BC9D9F5B8494A639586E4E137FAFD8D
2016-01-14 18:31:40,936 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 3BC9D9F5B8494A639586E4E137FAFD8D
2016-01-14 18:31:40,936 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@70c0c608; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@9710d; comparator=cascading.tuple.hadoop.util.TupleComparator@6bdd0c7f; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:40,936 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:40,937 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:40,937 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:40,951 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:40,951 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:40,951 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:40,951 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:40,951 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:40,952 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:40,952 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:40,952 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F8925C37892C4368A625E693EDAF478A, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:40,955 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796300724_0001_1_00_000000_0
2016-01-14 18:31:40,955 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F8925C37892C4368A625E693EDAF478A, mem on close (mb), free: 302, total: 562, max: 672
2016-01-14 18:31:40,955 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:40,955 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:40,955 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:40,955 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:40,957 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:40 Completed running task attempt: attempt_1452796300724_0001_1_00_000000_0
2016-01-14 18:31:40,959 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796300724_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:40,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796300724_0001_1_00, eventCount=1
2016-01-14 18:31:40,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:40,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F8925C37892C4368A625E693EDAF478A, taskAttemptId=attempt_1452796300724_0001_1_00_000000_0, startTime=1452796300915, finishTime=1452796300960, timeTaken=45, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796300952, Process_Duration=3, Process_End_Time=1452796300955, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:40,960 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300724_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:40,960 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:40,960 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796300724_0001_1_00_000000_0
2016-01-14 18:31:40,961 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:40,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:TASK_FINISHED]: vertexName=F8925C37892C4368A625E693EDAF478A, taskId=task_1452796300724_0001_1_00_000000, startTime=1452796300915, finishTime=1452796300960, timeTaken=45, status=SUCCEEDED, successfulAttemptID=attempt_1452796300724_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796300952, Process_Duration=3, Process_End_Time=1452796300955, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:40,961 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300724_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:40,961 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A] : 1
2016-01-14 18:31:40,961 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796300952, Process_Duration=3, Process_End_Time=1452796300955, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:40,961 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A]
2016-01-14 18:31:40,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:40,962 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:40,962 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:40,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:VERTEX_FINISHED]: vertexName=F8925C37892C4368A625E693EDAF478A, vertexId=vertex_1452796300724_0001_1_00, initRequestedTime=1452796300882, initedTime=1452796300894, startRequestedTime=1452796300895, startedTime=1452796300895, finishTime=1452796300961, timeTaken=66, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796300952, Process_Duration=3, Process_End_Time=1452796300955, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796300915, firstTasksToStart=[ task_1452796300724_0001_1_00_000000 ], lastTaskFinishTime=1452796300960, lastTasksToFinish=[ task_1452796300724_0001_1_00_000000 ], minTaskDuration=45, maxTaskDuration=45, avgTaskDuration=45.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796300724_0001_1_00_000000 ], longestDurationTasks=[ task_1452796300724_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:40,962 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796300724_0001_00_000001
2016-01-14 18:31:40,962 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:40,963 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:40,963 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796300724_0001_00_000001, stoppedTime=1452796300963, exitStatus=0
2016-01-14 18:31:40,963 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:40,963 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF] attempt: attempt_1452796300724_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:40,963 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:40,963 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: F39D92C0B67E4D54941368A5AF7BE5BF
2016-01-14 18:31:40,963 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF]
2016-01-14 18:31:40,964 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:40,964 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796300724_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:40,964 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:TASK_STARTED]: vertexName=F39D92C0B67E4D54941368A5AF7BE5BF, taskId=task_1452796300724_0001_1_01_000000, scheduledTime=1452796300964, launchTime=1452796300964
2016-01-14 18:31:40,964 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300724_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:40,964 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:40,964 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF]
2016-01-14 18:31:40,964 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: F39D92C0B67E4D54941368A5AF7BE5BF, VertexParallelism: 1, TaskAttemptID:attempt_1452796300724_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F8925C37892C4368A625E693EDAF478A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B65AC996999B452BB5FB696FAB17A812, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:40,965 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300724_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:40,965 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:40,965 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:40,965 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796300724_0001_00_000002
2016-01-14 18:31:40,965 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:40,967 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796300724_0001_00_000002
2016-01-14 18:31:40,967 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:40,968 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:40,968 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:40,968 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796300724_0001_00_000002, launchTime=1452796300968
2016-01-14 18:31:40,969 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796300724_0001_00_000002
2016-01-14 18:31:40,969 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796300724_0001_1_01_000000_0] to container: [container_1452796300724_0001_00_000002]
2016-01-14 18:31:40,969 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:40,970 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796300724_0001_1_01_000000_0] started. Is using containerId: [container_1452796300724_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:40,970 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:40,970 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796300724_0001_00_000002 given task: attempt_1452796300724_0001_1_01_000000_0
2016-01-14 18:31:40,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F39D92C0B67E4D54941368A5AF7BE5BF, taskAttemptId=attempt_1452796300724_0001_1_01_000000_0, startTime=1452796300969, containerId=container_1452796300724_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796300724_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:40,970 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300724_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:40,970 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796300724_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796300724_0001_1_01_000000_0
2016-01-14 18:31:40,970 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300724_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:40 Starting to run new task attempt: attempt_1452796300724_0001_1_01_000000_0
2016-01-14 18:31:40,970 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796300724_0001_1_01_000000_0
2016-01-14 18:31:40,971 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:40,971 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:40,971 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:40,971 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:40,971 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:40,971 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:40,972 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873] every/(1/1) ...platform/ALL_'field'_SWAP, VertexName: F39D92C0B67E4D54941368A5AF7BE5BF, VertexParallelism: 1, TaskAttemptID:attempt_1452796300724_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F8925C37892C4368A625E693EDAF478A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=B65AC996999B452BB5FB696FAB17A812, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:40,972 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:40,972 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796300724_0001_1_01_000000_0
2016-01-14 18:31:40,973 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796300724_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:40,973 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:40,972 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796300724_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:40,973 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796300724_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:40,973 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796300724_0001_00_000001, Exiting
2016-01-14 18:31:40,973 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796300724_0001_00_000001
2016-01-14 18:31:40,973 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796300724_0001_00_000001
2016-01-14 18:31:40,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,974 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,976 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796300724_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:40,977 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796300724_0001_00_000001 completed successfully
2016-01-14 18:31:40,977 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:40,978 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:40,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F8925C37892C4368A625E693EDAF478A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:40,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:40,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F8925C37892C4368A625E693EDAF478A
2016-01-14 18:31:40,978 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B65AC996999B452BB5FB696FAB17A812, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:40,979 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:40,980 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B65AC996999B452BB5FB696FAB17A812
2016-01-14 18:31:40,981 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:40,981 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: B65AC996999B452BB5FB696FAB17A812
2016-01-14 18:31:40,981 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:40,981 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:40,982 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F8925C37892C4368A625E693EDAF478A
2016-01-14 18:31:40,982 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F39D92C0B67E4D54941368A5AF7BE5BF, ordinal: 1
2016-01-14 18:31:40,982 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F39D92C0B67E4D54941368A5AF7BE5BF, mem on start (mb), free: 297, total: 562, max: 672
2016-01-14 18:31:40,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:40,982 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:40,983 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F8925C37892C4368A625E693EDAF478A
2016-01-14 18:31:40,983 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:40,988 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:40,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B65AC996999B452BB5FB696FAB17A812
2016-01-14 18:31:40,988 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:40,989 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:40,989 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:40,989 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:40,989 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:40,989 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:40,990 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:40,990 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B65AC996999B452BB5FB696FAB17A812, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:40,990 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F8925C37892C4368A625E693EDAF478A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:40,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:40,990 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F8925C37892C4368A625E693EDAF478A
2016-01-14 18:31:40,991 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:40,991 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:40,991 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F8925C37892C4368A625E693EDAF478A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:40,991 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:40,991 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:40,991 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:40,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F8925C37892C4368A625E693EDAF478A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:40,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:40,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:40,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F8925C37892C4368A625E693EDAF478A
2016-01-14 18:31:40,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:40,994 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796300724_0001_1_01_000000_0
2016-01-14 18:31:40,994 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796300724_0001_1_00_000000_0_10255, runDuration: 37514, ]
2016-01-14 18:31:40,995 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796300724_0001_1_00_000000_0_10255] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796300995, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:40,995 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F8925C37892C4368A625E693EDAF478A
2016-01-14 18:31:40,996 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:40,996 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F8925C37892C4368A625E693EDAF478A] #43 in 1ms
2016-01-14 18:31:40,996 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F8925C37892C4368A625E693EDAF478A] #43, status:false, isInterrupted:false
2016-01-14 18:31:40,997 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:40,997 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:40,997 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:40,997 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:40,999 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 3BC9D9F5B8494A639586E4E137FAFD8D
2016-01-14 18:31:40,999 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP"], id: B65AC996999B452BB5FB696FAB17A812
2016-01-14 18:31:40,999 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_'field'_SWAP"]
2016-01-14 18:31:40,999 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 3BC9D9F5B8494A639586E4E137FAFD8D, for 1 inputs
2016-01-14 18:31:41,000 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:41,000 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:41,000 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F39D92C0B67E4D54941368A5AF7BE5BF, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:41,000 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F8925C37892C4368A625E693EDAF478A
2016-01-14 18:31:41,001 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@32cd1288; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@357bb8c6
2016-01-14 18:31:41,003 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:41,003 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:41,004 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:41,004 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796300724_0001_1_01_000000_0
2016-01-14 18:31:41,004 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796300724_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:41,005 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796300724_0001_1_01_000000_0
2016-01-14 18:31:41,005 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F39D92C0B67E4D54941368A5AF7BE5BF, mem on close (mb), free: 294, total: 562, max: 672
2016-01-14 18:31:41,005 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F8925C37892C4368A625E693EDAF478A
2016-01-14 18:31:41,005 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:41,006 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:41,006 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:41,006 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:41,006 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:41 Completed running task attempt: attempt_1452796300724_0001_1_01_000000_0
2016-01-14 18:31:41,007 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796300724_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:41,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F39D92C0B67E4D54941368A5AF7BE5BF, taskAttemptId=attempt_1452796300724_0001_1_01_000000_0, startTime=1452796300969, finishTime=1452796301007, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796301000, Process_Duration=5, Process_End_Time=1452796301005, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:41,008 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796300724_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:41,008 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796300724_0001_1_01_000000_0
2016-01-14 18:31:41,008 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:TASK_FINISHED]: vertexName=F39D92C0B67E4D54941368A5AF7BE5BF, taskId=task_1452796300724_0001_1_01_000000, startTime=1452796300969, finishTime=1452796301008, timeTaken=39, status=SUCCEEDED, successfulAttemptID=attempt_1452796300724_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796301000, Process_Duration=5, Process_End_Time=1452796301005, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:41,008 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796300724_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:41,008 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF] : 1
2016-01-14 18:31:41,008 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF]
2016-01-14 18:31:41,009 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:41,009 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:VERTEX_FINISHED]: vertexName=F39D92C0B67E4D54941368A5AF7BE5BF, vertexId=vertex_1452796300724_0001_1_01, initRequestedTime=1452796300883, initedTime=1452796300890, startRequestedTime=1452796300897, startedTime=1452796300897, finishTime=1452796301008, timeTaken=111, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796301000, Process_Duration=5, Process_End_Time=1452796301005, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796300969, firstTasksToStart=[ task_1452796300724_0001_1_01_000000 ], lastTaskFinishTime=1452796301007, lastTasksToFinish=[ task_1452796300724_0001_1_01_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796300724_0001_1_01_000000 ], longestDurationTasks=[ task_1452796300724_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:41,009 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:41,009 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:41,010 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:41,010 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:41,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796301000, Process_Duration=5, Process_End_Time=1452796301005, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:41,011 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:41,011 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:41,011 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796300724_0001_00_000002
2016-01-14 18:31:41,011 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:41,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796300724_0001_00_000002, stoppedTime=1452796301013, exitStatus=0
2016-01-14 18:31:41,013 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:41,013 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:41,013 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796300724_0001_1
2016-01-14 18:31:41,013 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796300724_0001_1
2016-01-14 18:31:41,013 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796300724_0001_1_00 [F8925C37892C4368A625E693EDAF478A]
2016-01-14 18:31:41,014 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: B65AC996999B452BB5FB696FAB17A812 for vertex: vertex_1452796300724_0001_1_01 [F39D92C0B67E4D54941368A5AF7BE5BF]
2016-01-14 18:31:41,014 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:41,021 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796300724_0001_1, queueSize=0
2016-01-14 18:31:41,021 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:41,022 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796300724_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796300724_0001_1, startTime=1452796300882, finishTime=1452796301020, timeTaken=138, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1179648000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592601952, Process_Duration=8, Process_End_Time=2905592601960, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:41,022 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796300724_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:41,023 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796300724_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:41,023 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796300724_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:41,023 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796300724_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:41,023 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796300724_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:41,023 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796300724_0001_00_000002, Exiting
2016-01-14 18:31:41,023 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796300724_0001_00_000002
2016-01-14 18:31:41,023 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796300724_0001_00_000002
2016-01-14 18:31:41 Completed Dag: dag_1452796300724_0001_1
2016-01-14 18:31:41,024 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796300724_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:41,024 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796300724_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:41,024 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796300724_0001_00_000002 completed successfully
2016-01-14 18:31:41,024 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796300724_0001_1_post
2016-01-14 18:31:41,024 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:41,025 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:41,025 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:41,026 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796300724_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:41,030 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_'field'_SWAP, applicationId=application_1452796300724_0001
2016-01-14 18:31:41,032 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:41,032 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:41,032 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:41,032 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:41,032 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:41,032 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:41,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:41,033 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:41,033 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:41,033 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:41,033 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:41,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:41,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:41,038 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/30B3AB4E66784DA094B3C4D2E0C17B2E/ECBC106B2E39484CAD6855A53071D873/.tez/application_1452796300724_0001
2016-01-14 18:31:41,038 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.332
2016-01-14 18:31:41,110 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_'field'_SWAP
2016-01-14 18:31:41,137 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:41,137 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:41,166 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 18:31:41,171 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 18:31:41,171 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:41,171 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:41,171 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:41,184 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:41,188 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1
2016-01-14 18:31:41,230 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:41,230 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:41,231 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1"]
2016-01-14 18:31:41,231 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:41,231 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:41,232 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:41,232 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-01-14 18:31:41,232 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:41,243 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:41,248 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:41,248 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:41,248 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:41,256 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B/.tez/application_1452796301248_0001 doesn't exist and is created
2016-01-14 18:31:41,270 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:41,270 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:41,270 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B/.tez/application_1452796301248_0001
2016-01-14 18:31:41,271 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796301248_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:41,281 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:41,282 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:41,283 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:41,283 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:41,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B/.tez/application_1452796301248_0001/localmode-log-dir
2016-01-14 18:31:41,283 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B/.tez/application_1452796301248_0001/localmode-log-dir/history.txt.appattempt_1452796301248_0001_000000, maxErrors=10
2016-01-14 18:31:41,283 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:41,283 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796301248_0001_000000, appSubmitTime=1452796301271, launchTime=1452796301271
2016-01-14 18:31:41,287 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:41,287 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:41,287 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:41,287 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:41,289 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:46316
2016-01-14 18:31:41,290 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:41,292 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796301248_0001_000000, startTime=1452796301292
2016-01-14 18:31:41,292 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:41,292 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:41,309 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:41,309 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:41,371 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:41,371 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:41,371 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1452796301248_0001, dagName=[303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-01-14 18:31:41,387 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-01-14 18:31:41,388 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796301248_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B/.tez/application_1452796301248_0001/localmode-log-dir/dag_1452796301248_0001_1.dot
2016-01-14 18:31:41,389 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B/.tez/application_1452796301248_0001/localmode-log-dir/dag_1452796301248_0001_1-tez-dag.pb.txt
2016-01-14 18:31:41,395 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796301248_0001_1
2016-01-14 18:31:41,395 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:41,395 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-01-14 18:31:41 Running Dag: dag_1452796301248_0001_1
2016-01-14 18:31:41,404 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:41,404 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796301248_0001_1, submitTime=1452796301387
2016-01-14 18:31:41,405 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FD10391645854D4984A861E682890E43
2016-01-14 18:31:41,405 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:41,405 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:41,405 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796301248_0001_1, initTime=1452796301404
2016-01-14 18:31:41,406 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796301248_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:41,406 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:DAG_STARTED]: dagID=dag_1452796301248_0001_1, startTime=1452796301406
2016-01-14 18:31:41,406 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:41,406 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:41,406 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796301248_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:41,406 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A17E4D0211F141938956CB1B0B829BA9 : {32D2C5C9D8D245E080CFDA0BD5F1BA9D={InputName=32D2C5C9D8D245E080CFDA0BD5F1BA9D}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:41,406 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 32D2C5C9D8D245E080CFDA0BD5F1BA9D, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:41,406 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9]
2016-01-14 18:31:41,407 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9]
2016-01-14 18:31:41,407 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9]
2016-01-14 18:31:41,407 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9]. Starting root input initializers: 1
2016-01-14 18:31:41,407 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1452796301248_0001, dagName=[303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1
2016-01-14 18:31:41,407 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:41,407 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43]
2016-01-14 18:31:41,408 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 32D2C5C9D8D245E080CFDA0BD5F1BA9D on vertex vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9]
2016-01-14 18:31:41,410 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:41,410 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 32D2C5C9D8D245E080CFDA0BD5F1BA9D asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:41,410 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:41,411 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43]
2016-01-14 18:31:41,411 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:41,411 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43]
2016-01-14 18:31:41,411 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43]
2016-01-14 18:31:41,411 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=35BF285D42954EA496FFEAC5A72F6242, vertexId=vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:41,411 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=35BF285D42954EA496FFEAC5A72F6242, vertexId=vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43]
2016-01-14 18:31:41,413 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=35BF285D42954EA496FFEAC5A72F6242, vertexName=FD10391645854D4984A861E682890E43, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:41,413 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=35BF285D42954EA496FFEAC5A72F6242, vertexId=vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43]
2016-01-14 18:31:41,411 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:41,415 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:41,415 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FD10391645854D4984A861E682890E43, vertexId=vertex_1452796301248_0001_1_01, initRequestedTime=1452796301407, initedTime=1452796301415, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:41,416 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:41,416 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:41,416 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 32D2C5C9D8D245E080CFDA0BD5F1BA9D on vertex vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9]
2016-01-14 18:31:41,416 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:41,417 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {32D2C5C9D8D245E080CFDA0BD5F1BA9D=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:41,417 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9] parallelism set to 1
2016-01-14 18:31:41,417 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:41,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796301248_0001_1_00, eventCount=1
2016-01-14 18:31:41,417 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A17E4D0211F141938956CB1B0B829BA9, vertexId=vertex_1452796301248_0001_1_00, initRequestedTime=1452796301406, initedTime=1452796301417, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:41,418 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9] with distanceFromRoot: 0
2016-01-14 18:31:41,418 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:41,418 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:41,418 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A17E4D0211F141938956CB1B0B829BA9
2016-01-14 18:31:41,418 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9]
2016-01-14 18:31:41,419 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796301248_0001_1_00, startRequestedTime=1452796301418, startedTime=1452796301418
2016-01-14 18:31:41,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:41,419 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:41,419 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:41,419 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:TASK_STARTED]: vertexName=A17E4D0211F141938956CB1B0B829BA9, taskId=task_1452796301248_0001_1_00_000000, scheduledTime=1452796301419, launchTime=1452796301419
2016-01-14 18:31:41,420 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301248_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:41,420 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796301248_0001_1_00 for vertex: vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43] numStartedSources: 1 numSources: 1
2016-01-14 18:31:41,420 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43] with distanceFromRoot: 1
2016-01-14 18:31:41,420 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9]
2016-01-14 18:31:41,420 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A17E4D0211F141938956CB1B0B829BA9 in vertex: FD10391645854D4984A861E682890E43
2016-01-14 18:31:41,421 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FD10391645854D4984A861E682890E43 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:41,421 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796301248_0001_1_01, startRequestedTime=1452796301420, startedTime=1452796301420
2016-01-14 18:31:41,421 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:41,421 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:41,421 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: A17E4D0211F141938956CB1B0B829BA9, VertexParallelism: 1, TaskAttemptID:attempt_1452796301248_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=32D2C5C9D8D245E080CFDA0BD5F1BA9D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FD10391645854D4984A861E682890E43, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:41,422 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301248_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:41,422 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:41,422 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:41,423 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:41,423 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:41,423 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796301248_0001_00_000001
2016-01-14 18:31:41,423 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:41,425 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796301248_0001, with dag id: dag_1452796301248_0001_1
2016-01-14 18:31:41,425 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796301248_0001_00_000001
2016-01-14 18:31:41,425 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:41,426 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:41,427 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:41,427 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796301248_0001_00_000001, launchTime=1452796301426
2016-01-14 18:31:41,428 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796301248_0001_00_000001
2016-01-14 18:31:41,428 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796301248_0001_1_00_000000_0] to container: [container_1452796301248_0001_00_000001]
2016-01-14 18:31:41,428 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:41,429 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796301248_0001_1_00_000000_0] started. Is using containerId: [container_1452796301248_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:41,429 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:41,429 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A17E4D0211F141938956CB1B0B829BA9, taskAttemptId=attempt_1452796301248_0001_1_00_000000_0, startTime=1452796301429, containerId=container_1452796301248_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796301248_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:41,429 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301248_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:41,429 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301248_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:41,430 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796301248_0001_00_000001 given task: attempt_1452796301248_0001_1_00_000000_0
2016-01-14 18:31:41,430 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796301248_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796301248_0001_1_00_000000_0
2016-01-14 18:31:41 Starting to run new task attempt: attempt_1452796301248_0001_1_00_000000_0
2016-01-14 18:31:41,430 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796301248_0001_1_00_000000_0
2016-01-14 18:31:41,430 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:41,431 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:41,431 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:41,431 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:41,431 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:41,431 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:41,431 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: A17E4D0211F141938956CB1B0B829BA9, VertexParallelism: 1, TaskAttemptID:attempt_1452796301248_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=32D2C5C9D8D245E080CFDA0BD5F1BA9D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FD10391645854D4984A861E682890E43, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:41,432 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:41,432 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796301248_0001_1_00_000000_0
2016-01-14 18:31:41,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:41,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:41,432 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:41,433 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796301248_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:41,433 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=32D2C5C9D8D245E080CFDA0BD5F1BA9D, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:41,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:41,434 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 32D2C5C9D8D245E080CFDA0BD5F1BA9D
2016-01-14 18:31:41,434 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 32D2C5C9D8D245E080CFDA0BD5F1BA9D
2016-01-14 18:31:41,435 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:41,437 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:41,437 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 32D2C5C9D8D245E080CFDA0BD5F1BA9D
2016-01-14 18:31:41,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 32D2C5C9D8D245E080CFDA0BD5F1BA9D
2016-01-14 18:31:41,437 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FD10391645854D4984A861E682890E43, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:41,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:41,438 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FD10391645854D4984A861E682890E43
2016-01-14 18:31:41,439 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A17E4D0211F141938956CB1B0B829BA9, ordinal: 0
2016-01-14 18:31:41,439 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A17E4D0211F141938956CB1B0B829BA9, mem on start (mb), free: 187, total: 562, max: 672
2016-01-14 18:31:41,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:41,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:41,439 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:41,440 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:41,440 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FD10391645854D4984A861E682890E43
2016-01-14 18:31:41,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FD10391645854D4984A861E682890E43
2016-01-14 18:31:41,440 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:41,440 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:41,441 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:41,441 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:41,441 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:41,441 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:41,441 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:41,441 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 32D2C5C9D8D245E080CFDA0BD5F1BA9D, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:41,442 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FD10391645854D4984A861E682890E43, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:41,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:41,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 32D2C5C9D8D245E080CFDA0BD5F1BA9D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:41,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:41,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:41,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 32D2C5C9D8D245E080CFDA0BD5F1BA9D
2016-01-14 18:31:41,443 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:41,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 32D2C5C9D8D245E080CFDA0BD5F1BA9D
2016-01-14 18:31:41,443 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:41,443 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796301248_0001_1_00_000000_0
2016-01-14 18:31:41,445 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:41,445 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@51ab0956
2016-01-14 18:31:41,446 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:41,446 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 32D2C5C9D8D245E080CFDA0BD5F1BA9D
2016-01-14 18:31:41,446 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 854682B7E03746748937F7DC919FFA9A
2016-01-14 18:31:41,446 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 854682B7E03746748937F7DC919FFA9A
2016-01-14 18:31:41,447 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@51ab0956
2016-01-14 18:31:41,447 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:41,447 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@257850d6; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7e0324d1; comparator=cascading.tuple.hadoop.util.TupleComparator@6655f9; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:41,447 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:41,447 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:41,447 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:41,447 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:41,467 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:41,468 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:41,468 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:41,468 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:41,468 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:41,469 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:41,469 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:41,469 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A17E4D0211F141938956CB1B0B829BA9, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:41,473 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796301248_0001_1_00_000000_0
2016-01-14 18:31:41,473 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A17E4D0211F141938956CB1B0B829BA9, mem on close (mb), free: 307, total: 562, max: 672
2016-01-14 18:31:41,473 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:41,473 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:41,473 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:41,473 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:41,476 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:41 Completed running task attempt: attempt_1452796301248_0001_1_00_000000_0
2016-01-14 18:31:41,479 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796301248_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:41,479 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796301248_0001_1_00, eventCount=1
2016-01-14 18:31:41,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796301469, Process_Duration=4, Process_End_Time=1452796301473, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:41,479 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:41,480 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:41,480 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A17E4D0211F141938956CB1B0B829BA9, taskAttemptId=attempt_1452796301248_0001_1_00_000000_0, startTime=1452796301429, finishTime=1452796301479, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796301469, Process_Duration=4, Process_End_Time=1452796301473, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:41,480 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:41,480 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301248_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:41,480 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:41,480 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796301248_0001_00_000001
2016-01-14 18:31:41,480 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796301248_0001_1_00_000000_0
2016-01-14 18:31:41,481 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:41,481 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:TASK_FINISHED]: vertexName=A17E4D0211F141938956CB1B0B829BA9, taskId=task_1452796301248_0001_1_00_000000, startTime=1452796301429, finishTime=1452796301480, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1452796301248_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796301469, Process_Duration=4, Process_End_Time=1452796301473, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:41,481 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301248_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:41,481 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9] : 1
2016-01-14 18:31:41,481 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9]
2016-01-14 18:31:41,481 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:41,481 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:VERTEX_FINISHED]: vertexName=A17E4D0211F141938956CB1B0B829BA9, vertexId=vertex_1452796301248_0001_1_00, initRequestedTime=1452796301406, initedTime=1452796301417, startRequestedTime=1452796301418, startedTime=1452796301418, finishTime=1452796301481, timeTaken=63, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796301469, Process_Duration=4, Process_End_Time=1452796301473, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796301429, firstTasksToStart=[ task_1452796301248_0001_1_00_000000 ], lastTaskFinishTime=1452796301479, lastTasksToFinish=[ task_1452796301248_0001_1_00_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796301248_0001_1_00_000000 ], longestDurationTasks=[ task_1452796301248_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:41,482 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:41,482 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:41,482 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796301248_0001_00_000001, stoppedTime=1452796301482, exitStatus=0
2016-01-14 18:31:41,482 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:41,482 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:41,482 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43] attempt: attempt_1452796301248_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:41,482 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FD10391645854D4984A861E682890E43
2016-01-14 18:31:41,482 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43]
2016-01-14 18:31:41,483 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:41,483 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796301248_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:41,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:TASK_STARTED]: vertexName=FD10391645854D4984A861E682890E43, taskId=task_1452796301248_0001_1_01_000000, scheduledTime=1452796301483, launchTime=1452796301483
2016-01-14 18:31:41,483 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301248_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:41,483 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:41,484 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43]
2016-01-14 18:31:41,484 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: FD10391645854D4984A861E682890E43, VertexParallelism: 1, TaskAttemptID:attempt_1452796301248_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A17E4D0211F141938956CB1B0B829BA9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=35BF285D42954EA496FFEAC5A72F6242, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:41,484 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301248_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:41,483 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796301248_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:41,483 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:41,485 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796301248_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:41,485 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:41,485 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796301248_0001_00_000001, Exiting
2016-01-14 18:31:41,485 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796301248_0001_00_000001
2016-01-14 18:31:41,485 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796301248_0001_00_000001
2016-01-14 18:31:41,486 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796301248_0001_00_000002
2016-01-14 18:31:41,486 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:41,486 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796301248_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:41,486 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796301248_0001_00_000001 completed successfully
2016-01-14 18:31:41,487 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:41,487 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796301248_0001_00_000002
2016-01-14 18:31:41,488 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:41,489 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:41,489 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:41,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796301248_0001_00_000002, launchTime=1452796301489
2016-01-14 18:31:41,490 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796301248_0001_00_000002
2016-01-14 18:31:41,491 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796301248_0001_1_01_000000_0] to container: [container_1452796301248_0001_00_000002]
2016-01-14 18:31:41,491 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:41,491 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796301248_0001_1_01_000000_0] started. Is using containerId: [container_1452796301248_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:41,491 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FD10391645854D4984A861E682890E43, taskAttemptId=attempt_1452796301248_0001_1_01_000000_0, startTime=1452796301491, containerId=container_1452796301248_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796301248_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:41,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:41,491 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301248_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:41,492 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301248_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:41,492 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796301248_0001_00_000002 given task: attempt_1452796301248_0001_1_01_000000_0
2016-01-14 18:31:41,492 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796301248_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796301248_0001_1_01_000000_0
2016-01-14 18:31:41 Starting to run new task attempt: attempt_1452796301248_0001_1_01_000000_0
2016-01-14 18:31:41,492 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796301248_0001_1_01_000000_0
2016-01-14 18:31:41,492 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:41,492 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:41,492 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:41,493 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:41,493 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:41,493 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:41,493 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B] every/(1/1) ...lyplatform/ALL_UNKNOWN_-1, VertexName: FD10391645854D4984A861E682890E43, VertexParallelism: 1, TaskAttemptID:attempt_1452796301248_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A17E4D0211F141938956CB1B0B829BA9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=35BF285D42954EA496FFEAC5A72F6242, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:41,493 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:41,494 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796301248_0001_1_01_000000_0
2016-01-14 18:31:41,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:41,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:41,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:41,494 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A17E4D0211F141938956CB1B0B829BA9, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:41,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:41,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A17E4D0211F141938956CB1B0B829BA9
2016-01-14 18:31:41,496 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:41,497 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:41,497 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:41,497 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A17E4D0211F141938956CB1B0B829BA9
2016-01-14 18:31:41,497 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A17E4D0211F141938956CB1B0B829BA9
2016-01-14 18:31:41,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=35BF285D42954EA496FFEAC5A72F6242, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:41,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:41,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 35BF285D42954EA496FFEAC5A72F6242
2016-01-14 18:31:41,498 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:41,499 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 35BF285D42954EA496FFEAC5A72F6242
2016-01-14 18:31:41,499 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796301248_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:41,501 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FD10391645854D4984A861E682890E43, ordinal: 1
2016-01-14 18:31:41,501 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD10391645854D4984A861E682890E43, mem on start (mb), free: 301, total: 562, max: 672
2016-01-14 18:31:41,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:41,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:41,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:41,506 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:41,506 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 35BF285D42954EA496FFEAC5A72F6242
2016-01-14 18:31:41,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:41,507 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:41,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:41,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:41,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:41,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:41,507 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:41,508 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A17E4D0211F141938956CB1B0B829BA9, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:41,508 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 35BF285D42954EA496FFEAC5A72F6242, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:41,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:41,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A17E4D0211F141938956CB1B0B829BA9 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:41,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:41,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:41,508 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A17E4D0211F141938956CB1B0B829BA9
2016-01-14 18:31:41,508 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:41,509 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:41,510 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A17E4D0211F141938956CB1B0B829BA9 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:41,510 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:41,510 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:41,510 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:41,511 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A17E4D0211F141938956CB1B0B829BA9
2016-01-14 18:31:41,511 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:41,511 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796301248_0001_1_01_000000_0
2016-01-14 18:31:41,512 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796301248_0001_1_00_000000_0_10261, runDuration: 39477, ]
2016-01-14 18:31:41,514 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796301248_0001_1_00_000000_0_10261] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796301514, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:41,514 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A17E4D0211F141938956CB1B0B829BA9
2016-01-14 18:31:41,514 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:41,515 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:41,515 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 854682B7E03746748937F7DC919FFA9A
2016-01-14 18:31:41,515 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1"], id: 35BF285D42954EA496FFEAC5A72F6242
2016-01-14 18:31:41,516 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_-1"]
2016-01-14 18:31:41,516 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 854682B7E03746748937F7DC919FFA9A, for 1 inputs
2016-01-14 18:31:41,515 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A17E4D0211F141938956CB1B0B829BA9] #44 in 2ms
2016-01-14 18:31:41,516 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A17E4D0211F141938956CB1B0B829BA9] #44, status:false, isInterrupted:false
2016-01-14 18:31:41,517 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:41,517 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:41,517 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:41,517 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:41,517 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:41,517 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A17E4D0211F141938956CB1B0B829BA9
2016-01-14 18:31:41,517 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:41,518 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FD10391645854D4984A861E682890E43, all 1 inputs ready in: 00:00:00.002
2016-01-14 18:31:41,518 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@61855cd; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@27a908db
2016-01-14 18:31:41,523 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:41,523 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:41,524 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796301248_0001_1_01_000000_0
2016-01-14 18:31:41,524 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796301248_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:41,525 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796301248_0001_1_01_000000_0
2016-01-14 18:31:41,525 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FD10391645854D4984A861E682890E43, mem on close (mb), free: 298, total: 562, max: 672
2016-01-14 18:31:41,526 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A17E4D0211F141938956CB1B0B829BA9
2016-01-14 18:31:41,526 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:41,526 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:41,526 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:41,527 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:41,527 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:41 Completed running task attempt: attempt_1452796301248_0001_1_01_000000_0
2016-01-14 18:31:41,528 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796301248_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:41,528 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:41,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FD10391645854D4984A861E682890E43, taskAttemptId=attempt_1452796301248_0001_1_01_000000_0, startTime=1452796301491, finishTime=1452796301528, timeTaken=37, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796301518, Process_Duration=7, Process_End_Time=1452796301525, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:41,529 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301248_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:41,529 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796301248_0001_1_01_000000_0
2016-01-14 18:31:41,529 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:41,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:41,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:TASK_FINISHED]: vertexName=FD10391645854D4984A861E682890E43, taskId=task_1452796301248_0001_1_01_000000, startTime=1452796301491, finishTime=1452796301529, timeTaken=38, status=SUCCEEDED, successfulAttemptID=attempt_1452796301248_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796301518, Process_Duration=7, Process_End_Time=1452796301525, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:41,529 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301248_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:41,529 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43] : 1
2016-01-14 18:31:41,530 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43]
2016-01-14 18:31:41,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:41,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:VERTEX_FINISHED]: vertexName=FD10391645854D4984A861E682890E43, vertexId=vertex_1452796301248_0001_1_01, initRequestedTime=1452796301407, initedTime=1452796301415, startRequestedTime=1452796301420, startedTime=1452796301420, finishTime=1452796301530, timeTaken=110, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796301518, Process_Duration=7, Process_End_Time=1452796301525, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796301491, firstTasksToStart=[ task_1452796301248_0001_1_01_000000 ], lastTaskFinishTime=1452796301528, lastTasksToFinish=[ task_1452796301248_0001_1_01_000000 ], minTaskDuration=37, maxTaskDuration=37, avgTaskDuration=37.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796301248_0001_1_01_000000 ], longestDurationTasks=[ task_1452796301248_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:41,530 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:41,531 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:41,532 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796301248_0001_00_000002, stoppedTime=1452796301532, exitStatus=0
2016-01-14 18:31:41,532 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:41,532 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:41,532 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:41,532 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796301248_0001_1
2016-01-14 18:31:41,532 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796301248_0001_1
2016-01-14 18:31:41,532 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: 35BF285D42954EA496FFEAC5A72F6242 for vertex: vertex_1452796301248_0001_1_01 [FD10391645854D4984A861E682890E43]
2016-01-14 18:31:41,534 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=589824000, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796301518, Process_Duration=7, Process_End_Time=1452796301525, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:41,534 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:41,534 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:41,534 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796301248_0001_00_000002
2016-01-14 18:31:41,535 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796301248_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:41,535 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796301248_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:41,535 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796301248_0001_00_000002, Exiting
2016-01-14 18:31:41,535 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796301248_0001_00_000002
2016-01-14 18:31:41,535 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796301248_0001_00_000002
2016-01-14 18:31:41,535 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796301248_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:41,535 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796301248_0001_00_000002 completed successfully
2016-01-14 18:31:41,540 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796301248_0001_1_00 [A17E4D0211F141938956CB1B0B829BA9]
2016-01-14 18:31:41,541 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796301248_0001_1, queueSize=0
2016-01-14 18:31:41,541 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:41,541 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301248_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796301248_0001_1, startTime=1452796301406, finishTime=1452796301540, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1179648000, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=8, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592602987, Process_Duration=11, Process_End_Time=2905592602998, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:41,541 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796301248_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:41,541 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796301248_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:41,542 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796301248_0001_00_000002
2016-01-14 18:31:41,542 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301248_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:41 Completed Dag: dag_1452796301248_0001_1
2016-01-14 18:31:41,544 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796301248_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:41,544 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796301248_0001_1_post
2016-01-14 18:31:41,544 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:41,544 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:41,547 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_UNKNOWN_-1, applicationId=application_1452796301248_0001
2016-01-14 18:31:41,548 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:41,549 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:41,549 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:41,549 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:41,549 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:41,549 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:41,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:41,550 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:41,550 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:41,550 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:41,550 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:41,551 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:41,551 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:41,554 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/303522D6B51145CDBF6917713A653AC6/C8D626D1B8D241A88A5089FB57E3429B/.tez/application_1452796301248_0001
2016-01-14 18:31:41,555 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.324
2016-01-14 18:31:41,619 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_-1
2016-01-14 18:31:41,640 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:41,640 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:41,666 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-01-14 18:31:41,667 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-01-14 18:31:41,668 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:41,668 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:41,668 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_'field'
2016-01-14 18:31:41,689 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:41,689 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:41,722 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 18:31:41,725 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 18:31:41,725 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:41,725 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:41,726 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:41,738 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:41,743 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS
2016-01-14 18:31:41,786 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:41,786 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:41,786 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS"]
2016-01-14 18:31:41,786 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:41,786 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:41,787 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:41,787 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-01-14 18:31:41,787 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:41,798 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:41,802 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:41,803 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:41,803 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:41,812 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4/.tez/application_1452796301803_0001 doesn't exist and is created
2016-01-14 18:31:41,825 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:41,825 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:41,826 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4/.tez/application_1452796301803_0001
2016-01-14 18:31:41,827 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796301803_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:41,840 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:41,840 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:41,841 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:41,842 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:41,842 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4/.tez/application_1452796301803_0001/localmode-log-dir
2016-01-14 18:31:41,842 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4/.tez/application_1452796301803_0001/localmode-log-dir/history.txt.appattempt_1452796301803_0001_000000, maxErrors=10
2016-01-14 18:31:41,842 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:41,842 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796301803_0001_000000, appSubmitTime=1452796301827, launchTime=1452796301827
2016-01-14 18:31:41,848 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:45400
2016-01-14 18:31:41,848 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:41,848 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:41,849 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:41,849 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:41,850 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:41,851 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796301803_0001_000000, startTime=1452796301851
2016-01-14 18:31:41,851 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:41,851 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:41,925 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:41,926 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:41,926 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1452796301803_0001, dagName=[174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-01-14 18:31:41,941 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-01-14 18:31:41,942 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796301803_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4/.tez/application_1452796301803_0001/localmode-log-dir/dag_1452796301803_0001_1.dot
2016-01-14 18:31:41,943 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4/.tez/application_1452796301803_0001/localmode-log-dir/dag_1452796301803_0001_1-tez-dag.pb.txt
2016-01-14 18:31:41,949 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796301803_0001_1
2016-01-14 18:31:41,949 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:41,949 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-01-14 18:31:41 Running Dag: dag_1452796301803_0001_1
2016-01-14 18:31:41,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796301803_0001_1, submitTime=1452796301941
2016-01-14 18:31:41,959 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:41,959 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CD5CA7D5BC224517ACA82FE13BCFFD8F
2016-01-14 18:31:41,960 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:41,960 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796301803_0001_1, initTime=1452796301959
2016-01-14 18:31:41,960 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:41,961 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796301803_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:41,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:DAG_STARTED]: dagID=dag_1452796301803_0001_1, startTime=1452796301961
2016-01-14 18:31:41,961 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:41,961 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796301803_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:41,961 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F9779340CDBB44B08EAF754719991153 : {D2756EE459F649C085EB483AD09D9AAA={InputName=D2756EE459F649C085EB483AD09D9AAA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:41,961 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: D2756EE459F649C085EB483AD09D9AAA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:41,961 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153]
2016-01-14 18:31:41,961 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:41,961 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153]
2016-01-14 18:31:41,962 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153]
2016-01-14 18:31:41,962 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153]. Starting root input initializers: 1
2016-01-14 18:31:41,962 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1452796301803_0001, dagName=[174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS
2016-01-14 18:31:41,962 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:41,962 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F]
2016-01-14 18:31:41,964 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: D2756EE459F649C085EB483AD09D9AAA on vertex vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153]
2016-01-14 18:31:41,965 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:41,966 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F]
2016-01-14 18:31:41,966 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F]
2016-01-14 18:31:41,966 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F]
2016-01-14 18:31:41,966 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=FFF1444D2A764CA79C670150F4729236, vertexId=vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:41,966 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=FFF1444D2A764CA79C670150F4729236, vertexId=vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F]
2016-01-14 18:31:41,967 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:41,967 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input D2756EE459F649C085EB483AD09D9AAA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:41,969 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=FFF1444D2A764CA79C670150F4729236, vertexName=CD5CA7D5BC224517ACA82FE13BCFFD8F, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:41,969 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=FFF1444D2A764CA79C670150F4729236, vertexId=vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F]
2016-01-14 18:31:41,969 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:41,970 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:41,970 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:41,970 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CD5CA7D5BC224517ACA82FE13BCFFD8F, vertexId=vertex_1452796301803_0001_1_01, initRequestedTime=1452796301962, initedTime=1452796301970, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:41,970 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:41,972 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:41,972 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: D2756EE459F649C085EB483AD09D9AAA on vertex vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153]
2016-01-14 18:31:41,973 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:41,973 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {D2756EE459F649C085EB483AD09D9AAA=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:41,973 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153] parallelism set to 1
2016-01-14 18:31:41,973 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:41,973 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796301803_0001_1_00, eventCount=1
2016-01-14 18:31:41,974 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F9779340CDBB44B08EAF754719991153, vertexId=vertex_1452796301803_0001_1_00, initRequestedTime=1452796301961, initedTime=1452796301974, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:41,974 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153] with distanceFromRoot: 0
2016-01-14 18:31:41,974 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:41,974 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:41,974 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F9779340CDBB44B08EAF754719991153
2016-01-14 18:31:41,974 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153]
2016-01-14 18:31:41,975 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:41,975 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796301803_0001_1_00, startRequestedTime=1452796301974, startedTime=1452796301974
2016-01-14 18:31:41,975 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:41,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:41,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:TASK_STARTED]: vertexName=F9779340CDBB44B08EAF754719991153, taskId=task_1452796301803_0001_1_00_000000, scheduledTime=1452796301975, launchTime=1452796301975
2016-01-14 18:31:41,983 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301803_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:41,983 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796301803_0001_1_00 for vertex: vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F] numStartedSources: 1 numSources: 1
2016-01-14 18:31:41,983 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F] with distanceFromRoot: 1
2016-01-14 18:31:41,983 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153]
2016-01-14 18:31:41,984 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F9779340CDBB44B08EAF754719991153 in vertex: CD5CA7D5BC224517ACA82FE13BCFFD8F
2016-01-14 18:31:41,984 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CD5CA7D5BC224517ACA82FE13BCFFD8F with 1 source tasks and 1 pending tasks
2016-01-14 18:31:41,984 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:41,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796301803_0001_1_01, startRequestedTime=1452796301984, startedTime=1452796301984
2016-01-14 18:31:41,984 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:41,985 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: F9779340CDBB44B08EAF754719991153, VertexParallelism: 1, TaskAttemptID:attempt_1452796301803_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D2756EE459F649C085EB483AD09D9AAA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CD5CA7D5BC224517ACA82FE13BCFFD8F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:41,985 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301803_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:41,985 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:41,986 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:41,986 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:41,986 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:41,987 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796301803_0001_00_000001
2016-01-14 18:31:41,987 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:41,988 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796301803_0001, with dag id: dag_1452796301803_0001_1
2016-01-14 18:31:41,988 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796301803_0001_00_000001
2016-01-14 18:31:41,988 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:41,989 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:41,989 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:41,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796301803_0001_00_000001, launchTime=1452796301989
2016-01-14 18:31:41,990 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796301803_0001_00_000001
2016-01-14 18:31:41,990 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796301803_0001_1_00_000000_0] to container: [container_1452796301803_0001_00_000001]
2016-01-14 18:31:41,990 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:41,990 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796301803_0001_00_000001 given task: attempt_1452796301803_0001_1_00_000000_0
2016-01-14 18:31:41,991 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796301803_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796301803_0001_1_00_000000_0
2016-01-14 18:31:41,991 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796301803_0001_1_00_000000_0] started. Is using containerId: [container_1452796301803_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:41,991 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F9779340CDBB44B08EAF754719991153, taskAttemptId=attempt_1452796301803_0001_1_00_000000_0, startTime=1452796301991, containerId=container_1452796301803_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796301803_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:41,991 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301803_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:41 Starting to run new task attempt: attempt_1452796301803_0001_1_00_000000_0
2016-01-14 18:31:41,991 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796301803_0001_1_00_000000_0
2016-01-14 18:31:41,991 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:41,991 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301803_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:41,991 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:41,991 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:41,992 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:41,992 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:41,992 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:41,992 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: F9779340CDBB44B08EAF754719991153, VertexParallelism: 1, TaskAttemptID:attempt_1452796301803_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D2756EE459F649C085EB483AD09D9AAA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CD5CA7D5BC224517ACA82FE13BCFFD8F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:41,992 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:41,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:41,992 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796301803_0001_1_00_000000_0
2016-01-14 18:31:41,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:41,993 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:41,993 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796301803_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:41,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:41,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D2756EE459F649C085EB483AD09D9AAA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:41,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CD5CA7D5BC224517ACA82FE13BCFFD8F, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:41,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:41,994 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D2756EE459F649C085EB483AD09D9AAA
2016-01-14 18:31:41,995 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: D2756EE459F649C085EB483AD09D9AAA
2016-01-14 18:31:41,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:41,995 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CD5CA7D5BC224517ACA82FE13BCFFD8F
2016-01-14 18:31:41,997 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:41,997 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:41,997 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CD5CA7D5BC224517ACA82FE13BCFFD8F
2016-01-14 18:31:41,997 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CD5CA7D5BC224517ACA82FE13BCFFD8F
2016-01-14 18:31:41,997 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: D2756EE459F649C085EB483AD09D9AAA
2016-01-14 18:31:41,998 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D2756EE459F649C085EB483AD09D9AAA
2016-01-14 18:31:41,997 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:42,000 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F9779340CDBB44B08EAF754719991153, ordinal: 0
2016-01-14 18:31:42,000 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9779340CDBB44B08EAF754719991153, mem on start (mb), free: 400, total: 563, max: 672
2016-01-14 18:31:42,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:42,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:42,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:42,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:42,001 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:42,001 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:42,001 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:42,001 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:42,002 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:42,002 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:42,002 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D2756EE459F649C085EB483AD09D9AAA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:42,002 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CD5CA7D5BC224517ACA82FE13BCFFD8F, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:42,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:42,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D2756EE459F649C085EB483AD09D9AAA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:42,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D2756EE459F649C085EB483AD09D9AAA
2016-01-14 18:31:42,003 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:42,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D2756EE459F649C085EB483AD09D9AAA
2016-01-14 18:31:42,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:42,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:42,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:42,003 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796301803_0001_1_00_000000_0
2016-01-14 18:31:42,004 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:42,004 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4c474e9a
2016-01-14 18:31:42,005 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4c474e9a
2016-01-14 18:31:42,005 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:42,005 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:42,007 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:42,008 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: D2756EE459F649C085EB483AD09D9AAA
2016-01-14 18:31:42,008 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 1067C51AC9C242AB81255B11157C5CEA
2016-01-14 18:31:42,008 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 1067C51AC9C242AB81255B11157C5CEA
2016-01-14 18:31:42,008 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@797edeb7; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@784ffeb1; comparator=cascading.tuple.hadoop.util.TupleComparator@4bda542c; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:42,009 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:42,009 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:42,009 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:42,022 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:42,022 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:42,022 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:42,022 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:42,022 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:42,023 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:42,023 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:42,024 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F9779340CDBB44B08EAF754719991153, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:42,026 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796301803_0001_1_00_000000_0
2016-01-14 18:31:42,026 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F9779340CDBB44B08EAF754719991153, mem on close (mb), free: 298, total: 563, max: 672
2016-01-14 18:31:42,026 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:42,026 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:42,026 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:42,026 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:42,027 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:42 Completed running task attempt: attempt_1452796301803_0001_1_00_000000_0
2016-01-14 18:31:42,029 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796301803_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:42,030 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796302024, Process_Duration=2, Process_End_Time=1452796302026, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:42,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796301803_0001_1_00, eventCount=1
2016-01-14 18:31:42,030 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F9779340CDBB44B08EAF754719991153, taskAttemptId=attempt_1452796301803_0001_1_00_000000_0, startTime=1452796301991, finishTime=1452796302030, timeTaken=39, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302024, Process_Duration=2, Process_End_Time=1452796302026, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:42,031 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301803_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:42,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:42,031 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:42,031 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:42,031 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:42,031 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796301803_0001_1_00_000000_0
2016-01-14 18:31:42,031 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:42,031 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:TASK_FINISHED]: vertexName=F9779340CDBB44B08EAF754719991153, taskId=task_1452796301803_0001_1_00_000000, startTime=1452796301991, finishTime=1452796302031, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1452796301803_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302024, Process_Duration=2, Process_End_Time=1452796302026, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:42,032 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301803_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:42,032 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153] : 1
2016-01-14 18:31:42,032 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153]
2016-01-14 18:31:42,032 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:42,032 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:VERTEX_FINISHED]: vertexName=F9779340CDBB44B08EAF754719991153, vertexId=vertex_1452796301803_0001_1_00, initRequestedTime=1452796301961, initedTime=1452796301974, startRequestedTime=1452796301974, startedTime=1452796301974, finishTime=1452796302032, timeTaken=58, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302024, Process_Duration=2, Process_End_Time=1452796302026, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796301991, firstTasksToStart=[ task_1452796301803_0001_1_00_000000 ], lastTaskFinishTime=1452796302030, lastTasksToFinish=[ task_1452796301803_0001_1_00_000000 ], minTaskDuration=39, maxTaskDuration=39, avgTaskDuration=39.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796301803_0001_1_00_000000 ], longestDurationTasks=[ task_1452796301803_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:42,033 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:42,033 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:42,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796301803_0001_00_000001, stoppedTime=1452796302033, exitStatus=0
2016-01-14 18:31:42,033 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:42,033 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F] attempt: attempt_1452796301803_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:42,033 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:42,033 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CD5CA7D5BC224517ACA82FE13BCFFD8F
2016-01-14 18:31:42,033 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F]
2016-01-14 18:31:42,034 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:42,034 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796301803_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:42,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:TASK_STARTED]: vertexName=CD5CA7D5BC224517ACA82FE13BCFFD8F, taskId=task_1452796301803_0001_1_01_000000, scheduledTime=1452796302034, launchTime=1452796302034
2016-01-14 18:31:42,034 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301803_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:42,034 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:42,034 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F]
2016-01-14 18:31:42,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:42,035 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: CD5CA7D5BC224517ACA82FE13BCFFD8F, VertexParallelism: 1, TaskAttemptID:attempt_1452796301803_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9779340CDBB44B08EAF754719991153, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FFF1444D2A764CA79C670150F4729236, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:42,035 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301803_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:42,035 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:42,036 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796301803_0001_00_000002
2016-01-14 18:31:42,036 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:42,037 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796301803_0001_00_000001
2016-01-14 18:31:42,037 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796301803_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:42,037 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796301803_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:42,037 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796301803_0001_00_000002
2016-01-14 18:31:42,037 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:42,038 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:42,038 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796301803_0001_00_000001, Exiting
2016-01-14 18:31:42,038 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796301803_0001_00_000001
2016-01-14 18:31:42,038 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796301803_0001_00_000001
2016-01-14 18:31:42,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:42,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796301803_0001_00_000002, launchTime=1452796302037
2016-01-14 18:31:42,039 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796301803_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:42,039 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796301803_0001_00_000001 completed successfully
2016-01-14 18:31:42,039 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:42,040 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796301803_0001_00_000002
2016-01-14 18:31:42,040 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796301803_0001_1_01_000000_0] to container: [container_1452796301803_0001_00_000002]
2016-01-14 18:31:42,040 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:42,041 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796301803_0001_00_000002 given task: attempt_1452796301803_0001_1_01_000000_0
2016-01-14 18:31:42,041 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796301803_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796301803_0001_1_01_000000_0
2016-01-14 18:31:42,041 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796301803_0001_1_01_000000_0] started. Is using containerId: [container_1452796301803_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:42 Starting to run new task attempt: attempt_1452796301803_0001_1_01_000000_0
2016-01-14 18:31:42,041 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:42,041 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796301803_0001_1_01_000000_0
2016-01-14 18:31:42,042 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:42,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CD5CA7D5BC224517ACA82FE13BCFFD8F, taskAttemptId=attempt_1452796301803_0001_1_01_000000_0, startTime=1452796302041, containerId=container_1452796301803_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796301803_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:42,042 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:42,042 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:42,042 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301803_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:42,042 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:42,042 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:42,042 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301803_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:42,043 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:42,043 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4] every/(1/1) ...tform/ALL_UNKNOWN_RESULTS, VertexName: CD5CA7D5BC224517ACA82FE13BCFFD8F, VertexParallelism: 1, TaskAttemptID:attempt_1452796301803_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9779340CDBB44B08EAF754719991153, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=FFF1444D2A764CA79C670150F4729236, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:42,043 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:42,044 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796301803_0001_1_01_000000_0
2016-01-14 18:31:42,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:42,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:42,044 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796301803_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:42,044 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F9779340CDBB44B08EAF754719991153, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:42,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:42,045 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F9779340CDBB44B08EAF754719991153
2016-01-14 18:31:42,046 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:42,047 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:42,047 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F9779340CDBB44B08EAF754719991153
2016-01-14 18:31:42,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F9779340CDBB44B08EAF754719991153
2016-01-14 18:31:42,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:42,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FFF1444D2A764CA79C670150F4729236, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:42,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:42,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FFF1444D2A764CA79C670150F4729236
2016-01-14 18:31:42,049 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:42,049 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: FFF1444D2A764CA79C670150F4729236
2016-01-14 18:31:42,052 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:42,055 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CD5CA7D5BC224517ACA82FE13BCFFD8F, ordinal: 1
2016-01-14 18:31:42,055 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CD5CA7D5BC224517ACA82FE13BCFFD8F, mem on start (mb), free: 293, total: 563, max: 672
2016-01-14 18:31:42,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:42,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:42,056 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:42,058 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:42,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FFF1444D2A764CA79C670150F4729236
2016-01-14 18:31:42,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:42,058 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:42,058 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:42,058 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:42,058 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:42,058 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:42,059 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:42,059 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F9779340CDBB44B08EAF754719991153, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:42,059 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FFF1444D2A764CA79C670150F4729236, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:42,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:42,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F9779340CDBB44B08EAF754719991153
2016-01-14 18:31:42,059 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:42,060 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:42,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F9779340CDBB44B08EAF754719991153 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:42,060 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F9779340CDBB44B08EAF754719991153 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:42,060 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:42,060 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:42,060 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:42,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:42,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:42,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F9779340CDBB44B08EAF754719991153
2016-01-14 18:31:42,061 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:42,061 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796301803_0001_1_01_000000_0
2016-01-14 18:31:42,063 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:42,064 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796301803_0001_1_00_000000_0_10267, runDuration: 33288, ]
2016-01-14 18:31:42,064 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 1067C51AC9C242AB81255B11157C5CEA
2016-01-14 18:31:42,064 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS"], id: FFF1444D2A764CA79C670150F4729236
2016-01-14 18:31:42,064 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_RESULTS"]
2016-01-14 18:31:42,064 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 1067C51AC9C242AB81255B11157C5CEA, for 1 inputs
2016-01-14 18:31:42,065 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796301803_0001_1_00_000000_0_10267] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796302065, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:42,065 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F9779340CDBB44B08EAF754719991153
2016-01-14 18:31:42,066 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:42,066 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F9779340CDBB44B08EAF754719991153] #45 in 2ms
2016-01-14 18:31:42,066 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F9779340CDBB44B08EAF754719991153] #45, status:false, isInterrupted:false
2016-01-14 18:31:42,066 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:42,066 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:42,066 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:42,066 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:42,067 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:42,067 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F9779340CDBB44B08EAF754719991153
2016-01-14 18:31:42,067 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:42,067 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CD5CA7D5BC224517ACA82FE13BCFFD8F, all 1 inputs ready in: 00:00:00.003
2016-01-14 18:31:42,067 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6a79ccbe; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6271048e
2016-01-14 18:31:42,070 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:42,070 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:42,071 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796301803_0001_1_01_000000_0
2016-01-14 18:31:42,071 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796301803_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:42,071 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796301803_0001_1_01_000000_0
2016-01-14 18:31:42,071 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CD5CA7D5BC224517ACA82FE13BCFFD8F, mem on close (mb), free: 290, total: 563, max: 672
2016-01-14 18:31:42,072 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F9779340CDBB44B08EAF754719991153
2016-01-14 18:31:42,072 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:42,072 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:42,072 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:42,072 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:42,072 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:42 Completed running task attempt: attempt_1452796301803_0001_1_01_000000_0
2016-01-14 18:31:42,073 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796301803_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:42,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CD5CA7D5BC224517ACA82FE13BCFFD8F, taskAttemptId=attempt_1452796301803_0001_1_01_000000_0, startTime=1452796302041, finishTime=1452796302073, timeTaken=32, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302067, Process_Duration=4, Process_End_Time=1452796302071, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:42,074 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796301803_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:42,074 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:42,074 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796301803_0001_1_01_000000_0
2016-01-14 18:31:42,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:TASK_FINISHED]: vertexName=CD5CA7D5BC224517ACA82FE13BCFFD8F, taskId=task_1452796301803_0001_1_01_000000, startTime=1452796302041, finishTime=1452796302074, timeTaken=33, status=SUCCEEDED, successfulAttemptID=attempt_1452796301803_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302067, Process_Duration=4, Process_End_Time=1452796302071, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:42,074 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:42,075 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796301803_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:42,075 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F] : 1
2016-01-14 18:31:42,075 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F]
2016-01-14 18:31:42,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:42,075 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:42,075 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:VERTEX_FINISHED]: vertexName=CD5CA7D5BC224517ACA82FE13BCFFD8F, vertexId=vertex_1452796301803_0001_1_01, initRequestedTime=1452796301962, initedTime=1452796301970, startRequestedTime=1452796301984, startedTime=1452796301984, finishTime=1452796302075, timeTaken=91, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302067, Process_Duration=4, Process_End_Time=1452796302071, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796302041, firstTasksToStart=[ task_1452796301803_0001_1_01_000000 ], lastTaskFinishTime=1452796302073, lastTasksToFinish=[ task_1452796301803_0001_1_01_000000 ], minTaskDuration=32, maxTaskDuration=32, avgTaskDuration=32.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796301803_0001_1_01_000000 ], longestDurationTasks=[ task_1452796301803_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:42,076 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:42,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590348288, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796302067, Process_Duration=4, Process_End_Time=1452796302071, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:42,076 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:42,076 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:42,076 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:42,076 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796301803_0001_00_000002
2016-01-14 18:31:42,076 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796301803_0001_00_000002, stoppedTime=1452796302076, exitStatus=0
2016-01-14 18:31:42,076 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796301803_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:42,077 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796301803_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:42,077 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796301803_0001_00_000002, Exiting
2016-01-14 18:31:42,077 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796301803_0001_00_000002
2016-01-14 18:31:42,077 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796301803_0001_00_000002
2016-01-14 18:31:42,076 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:42,077 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796301803_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:42,077 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796301803_0001_00_000002 completed successfully
2016-01-14 18:31:42,077 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:42,077 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:42,077 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796301803_0001_1
2016-01-14 18:31:42,078 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796301803_0001_1
2016-01-14 18:31:42,078 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: FFF1444D2A764CA79C670150F4729236 for vertex: vertex_1452796301803_0001_1_01 [CD5CA7D5BC224517ACA82FE13BCFFD8F]
2016-01-14 18:31:42,084 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796301803_0001_1_00 [F9779340CDBB44B08EAF754719991153]
2016-01-14 18:31:42,085 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796301803_0001_1, queueSize=0
2016-01-14 18:31:42,085 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:42,086 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796301803_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796301803_0001_1, startTime=1452796301961, finishTime=1452796302084, timeTaken=123, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=985, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1180696576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=6, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592604091, Process_Duration=6, Process_End_Time=2905592604097, Read_Duration=1, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:42,086 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796301803_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:42,086 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796301803_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:42,086 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796301803_0001_00_000002
2016-01-14 18:31:42,086 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796301803_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:42 Completed Dag: dag_1452796301803_0001_1
2016-01-14 18:31:42,087 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796301803_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:42,087 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796301803_0001_1_post
2016-01-14 18:31:42,087 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:42,088 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:42,091 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_UNKNOWN_RESULTS, applicationId=application_1452796301803_0001
2016-01-14 18:31:42,093 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:42,093 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:42,093 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:42,093 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:42,093 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:42,093 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:42,093 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:42,094 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:42,094 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:42,094 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:42,094 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:42,094 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:42,095 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:42,098 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/174E14C5CA7B42CAB5FE78F773157FAD/DF693C44A85048A8A1239D5C8B1A8CB4/.tez/application_1452796301803_0001
2016-01-14 18:31:42,099 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.313
2016-01-14 18:31:42,169 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:42,169 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:42,172 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_RESULTS
2016-01-14 18:31:42,197 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:42,197 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:42,243 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 18:31:42,247 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 18:31:42,248 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:42,248 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:42,248 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:42,265 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:42,271 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL
2016-01-14 18:31:42,320 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:42,321 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:42,321 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL"]
2016-01-14 18:31:42,321 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:42,321 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:42,321 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:42,322 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-01-14 18:31:42,322 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:42,334 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:42,338 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:42,339 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:42,339 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:42,348 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1/.tez/application_1452796302339_0001 doesn't exist and is created
2016-01-14 18:31:42,361 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:42,361 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:42,362 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1/.tez/application_1452796302339_0001
2016-01-14 18:31:42,363 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796302339_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:42,373 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:42,373 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:42,374 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:42,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:42,374 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1/.tez/application_1452796302339_0001/localmode-log-dir
2016-01-14 18:31:42,374 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1/.tez/application_1452796302339_0001/localmode-log-dir/history.txt.appattempt_1452796302339_0001_000000, maxErrors=10
2016-01-14 18:31:42,374 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:42,375 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796302339_0001_000000, appSubmitTime=1452796302362, launchTime=1452796302363
2016-01-14 18:31:42,378 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:42,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:42,378 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:42,378 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:42,379 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:59963
2016-01-14 18:31:42,380 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:42,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796302339_0001_000000, startTime=1452796302382
2016-01-14 18:31:42,382 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:42,382 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:42,462 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:42,462 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:42,462 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1452796302339_0001, dagName=[E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-01-14 18:31:42,479 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-01-14 18:31:42,480 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796302339_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1/.tez/application_1452796302339_0001/localmode-log-dir/dag_1452796302339_0001_1.dot
2016-01-14 18:31:42,480 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1/.tez/application_1452796302339_0001/localmode-log-dir/dag_1452796302339_0001_1-tez-dag.pb.txt
2016-01-14 18:31:42,487 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796302339_0001_1
2016-01-14 18:31:42,487 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:42,487 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-01-14 18:31:42 Running Dag: dag_1452796302339_0001_1
2016-01-14 18:31:42,496 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796302339_0001_1, submitTime=1452796302479
2016-01-14 18:31:42,496 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:42,497 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B07E732BBDF9404B98DC2F69750C2EE5
2016-01-14 18:31:42,497 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:42,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796302339_0001_1, initTime=1452796302497
2016-01-14 18:31:42,497 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796302339_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:42,498 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:DAG_STARTED]: dagID=dag_1452796302339_0001_1, startTime=1452796302498
2016-01-14 18:31:42,498 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:42,498 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796302339_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:42,498 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C85853AB45884297B4F582A35FE26291 : {C1EC70563B714F70B4134DD79D432BE8={InputName=C1EC70563B714F70B4134DD79D432BE8}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:42,498 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: C1EC70563B714F70B4134DD79D432BE8, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:42,498 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291]
2016-01-14 18:31:42,498 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291]
2016-01-14 18:31:42,498 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291]
2016-01-14 18:31:42,499 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291]. Starting root input initializers: 1
2016-01-14 18:31:42,498 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:42,499 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:42,499 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1452796302339_0001, dagName=[E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL
2016-01-14 18:31:42,500 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:42,500 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5]
2016-01-14 18:31:42,500 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: C1EC70563B714F70B4134DD79D432BE8 on vertex vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291]
2016-01-14 18:31:42,502 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:42,502 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input C1EC70563B714F70B4134DD79D432BE8 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:42,503 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:42,503 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:42,503 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:42,504 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5]
2016-01-14 18:31:42,504 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5]
2016-01-14 18:31:42,504 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5]
2016-01-14 18:31:42,504 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=EA579CA08502432FB871DD843EF39EA1, vertexId=vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:42,504 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=EA579CA08502432FB871DD843EF39EA1, vertexId=vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5]
2016-01-14 18:31:42,506 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:42,506 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: C1EC70563B714F70B4134DD79D432BE8 on vertex vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291]
2016-01-14 18:31:42,507 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=EA579CA08502432FB871DD843EF39EA1, vertexName=B07E732BBDF9404B98DC2F69750C2EE5, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:42,507 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=EA579CA08502432FB871DD843EF39EA1, vertexId=vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5]
2016-01-14 18:31:42,508 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:42,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B07E732BBDF9404B98DC2F69750C2EE5, vertexId=vertex_1452796302339_0001_1_01, initRequestedTime=1452796302500, initedTime=1452796302508, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:42,508 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:42,508 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:42,508 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {C1EC70563B714F70B4134DD79D432BE8=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:42,509 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291] parallelism set to 1
2016-01-14 18:31:42,509 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:42,509 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796302339_0001_1_00, eventCount=1
2016-01-14 18:31:42,510 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C85853AB45884297B4F582A35FE26291, vertexId=vertex_1452796302339_0001_1_00, initRequestedTime=1452796302498, initedTime=1452796302510, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:42,511 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291] with distanceFromRoot: 0
2016-01-14 18:31:42,510 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:42,511 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:42,511 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C85853AB45884297B4F582A35FE26291
2016-01-14 18:31:42,512 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291]
2016-01-14 18:31:42,512 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796302339_0001_1_00, startRequestedTime=1452796302511, startedTime=1452796302511
2016-01-14 18:31:42,512 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:42,513 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:42,513 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:42,514 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:TASK_STARTED]: vertexName=C85853AB45884297B4F582A35FE26291, taskId=task_1452796302339_0001_1_00_000000, scheduledTime=1452796302513, launchTime=1452796302513
2016-01-14 18:31:42,514 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302339_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:42,514 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796302339_0001_1_00 for vertex: vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5] numStartedSources: 1 numSources: 1
2016-01-14 18:31:42,515 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5] with distanceFromRoot: 1
2016-01-14 18:31:42,515 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291]
2016-01-14 18:31:42,515 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796302339_0001, with dag id: dag_1452796302339_0001_1
2016-01-14 18:31:42,515 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C85853AB45884297B4F582A35FE26291 in vertex: B07E732BBDF9404B98DC2F69750C2EE5
2016-01-14 18:31:42,516 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B07E732BBDF9404B98DC2F69750C2EE5 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:42,516 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:42,516 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796302339_0001_1_01, startRequestedTime=1452796302515, startedTime=1452796302515
2016-01-14 18:31:42,516 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:42,516 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: C85853AB45884297B4F582A35FE26291, VertexParallelism: 1, TaskAttemptID:attempt_1452796302339_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C1EC70563B714F70B4134DD79D432BE8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B07E732BBDF9404B98DC2F69750C2EE5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:42,516 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302339_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:42,517 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:42,517 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:42,518 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:42,518 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:42,518 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796302339_0001_00_000001
2016-01-14 18:31:42,518 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:42,521 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796302339_0001_00_000001
2016-01-14 18:31:42,521 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:42,521 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:42,521 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:42,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796302339_0001_00_000001, launchTime=1452796302521
2016-01-14 18:31:42,522 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796302339_0001_00_000001
2016-01-14 18:31:42,522 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796302339_0001_1_00_000000_0] to container: [container_1452796302339_0001_00_000001]
2016-01-14 18:31:42,522 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:42,522 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796302339_0001_00_000001 given task: attempt_1452796302339_0001_1_00_000000_0
2016-01-14 18:31:42,522 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796302339_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796302339_0001_1_00_000000_0
2016-01-14 18:31:42 Starting to run new task attempt: attempt_1452796302339_0001_1_00_000000_0
2016-01-14 18:31:42,523 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796302339_0001_1_00_000000_0
2016-01-14 18:31:42,523 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:42,523 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:42,523 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:42,523 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:42,523 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:42,523 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:42,524 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: C85853AB45884297B4F582A35FE26291, VertexParallelism: 1, TaskAttemptID:attempt_1452796302339_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C1EC70563B714F70B4134DD79D432BE8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B07E732BBDF9404B98DC2F69750C2EE5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:42,524 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:42,524 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796302339_0001_1_00_000000_0] started. Is using containerId: [container_1452796302339_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:42,524 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796302339_0001_1_00_000000_0
2016-01-14 18:31:42,524 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:42,524 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:42,525 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796302339_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:42,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C1EC70563B714F70B4134DD79D432BE8, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:42,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C85853AB45884297B4F582A35FE26291, taskAttemptId=attempt_1452796302339_0001_1_00_000000_0, startTime=1452796302523, containerId=container_1452796302339_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796302339_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:42,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:42,525 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C1EC70563B714F70B4134DD79D432BE8
2016-01-14 18:31:42,525 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:42,526 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: C1EC70563B714F70B4134DD79D432BE8
2016-01-14 18:31:42,526 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302339_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:42,526 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302339_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:42,527 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:42,527 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: C1EC70563B714F70B4134DD79D432BE8
2016-01-14 18:31:42,527 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C1EC70563B714F70B4134DD79D432BE8
2016-01-14 18:31:42,529 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:42,531 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:42,532 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B07E732BBDF9404B98DC2F69750C2EE5, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:42,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:42,533 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B07E732BBDF9404B98DC2F69750C2EE5
2016-01-14 18:31:42,534 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C85853AB45884297B4F582A35FE26291, ordinal: 0
2016-01-14 18:31:42,534 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C85853AB45884297B4F582A35FE26291, mem on start (mb), free: 181, total: 563, max: 672
2016-01-14 18:31:42,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:42,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:42,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:42,535 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:42,536 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B07E732BBDF9404B98DC2F69750C2EE5
2016-01-14 18:31:42,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B07E732BBDF9404B98DC2F69750C2EE5
2016-01-14 18:31:42,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:42,536 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:42,536 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:42,536 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:42,537 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:42,537 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:42,537 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:42,537 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C1EC70563B714F70B4134DD79D432BE8, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:42,537 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B07E732BBDF9404B98DC2F69750C2EE5, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:42,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:42,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C1EC70563B714F70B4134DD79D432BE8 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:42,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:42,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:42,538 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C1EC70563B714F70B4134DD79D432BE8
2016-01-14 18:31:42,539 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:42,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C1EC70563B714F70B4134DD79D432BE8
2016-01-14 18:31:42,539 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:42,539 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796302339_0001_1_00_000000_0
2016-01-14 18:31:42,539 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:42,540 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@36bbb8c4
2016-01-14 18:31:42,541 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@36bbb8c4
2016-01-14 18:31:42,541 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:42,541 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:42,542 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:42,543 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: C1EC70563B714F70B4134DD79D432BE8
2016-01-14 18:31:42,543 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: BD170B8904434947B2ED9E1C012BCAEE
2016-01-14 18:31:42,543 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] BD170B8904434947B2ED9E1C012BCAEE
2016-01-14 18:31:42,543 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2c3c46ba; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@29477406; comparator=cascading.tuple.hadoop.util.TupleComparator@12124b16; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:42,544 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:42,544 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:42,544 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:42,564 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:42,565 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:42,565 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:42,565 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:42,565 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:42,566 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:42,566 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:42,566 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C85853AB45884297B4F582A35FE26291, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:42,569 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796302339_0001_1_00_000000_0
2016-01-14 18:31:42,569 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C85853AB45884297B4F582A35FE26291, mem on close (mb), free: 304, total: 564, max: 672
2016-01-14 18:31:42,569 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:42,569 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:42,569 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:42,569 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:42,570 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:42 Completed running task attempt: attempt_1452796302339_0001_1_00_000000_0
2016-01-14 18:31:42,573 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796302339_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:42,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796302566, Process_Duration=3, Process_End_Time=1452796302569, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:42,573 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796302339_0001_1_00, eventCount=1
2016-01-14 18:31:42,573 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:42,573 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:42,573 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796302339_0001_00_000001
2016-01-14 18:31:42,573 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C85853AB45884297B4F582A35FE26291, taskAttemptId=attempt_1452796302339_0001_1_00_000000_0, startTime=1452796302523, finishTime=1452796302573, timeTaken=50, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302566, Process_Duration=3, Process_End_Time=1452796302569, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:42,574 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302339_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:42,574 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:42,574 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796302339_0001_1_00_000000_0
2016-01-14 18:31:42,574 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:42,574 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:TASK_FINISHED]: vertexName=C85853AB45884297B4F582A35FE26291, taskId=task_1452796302339_0001_1_00_000000, startTime=1452796302523, finishTime=1452796302574, timeTaken=51, status=SUCCEEDED, successfulAttemptID=attempt_1452796302339_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302566, Process_Duration=3, Process_End_Time=1452796302569, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:42,574 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302339_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:42,574 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:42,574 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291] : 1
2016-01-14 18:31:42,575 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291]
2016-01-14 18:31:42,575 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:42,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:VERTEX_FINISHED]: vertexName=C85853AB45884297B4F582A35FE26291, vertexId=vertex_1452796302339_0001_1_00, initRequestedTime=1452796302498, initedTime=1452796302510, startRequestedTime=1452796302511, startedTime=1452796302511, finishTime=1452796302575, timeTaken=64, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302566, Process_Duration=3, Process_End_Time=1452796302569, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796302523, firstTasksToStart=[ task_1452796302339_0001_1_00_000000 ], lastTaskFinishTime=1452796302573, lastTasksToFinish=[ task_1452796302339_0001_1_00_000000 ], minTaskDuration=50, maxTaskDuration=50, avgTaskDuration=50.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796302339_0001_1_00_000000 ], longestDurationTasks=[ task_1452796302339_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:42,575 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:42,575 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:42,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796302339_0001_00_000001, stoppedTime=1452796302575, exitStatus=0
2016-01-14 18:31:42,575 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:42,576 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:42,576 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5] attempt: attempt_1452796302339_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:42,576 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B07E732BBDF9404B98DC2F69750C2EE5
2016-01-14 18:31:42,576 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5]
2016-01-14 18:31:42,576 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:42,576 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796302339_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:42,576 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:TASK_STARTED]: vertexName=B07E732BBDF9404B98DC2F69750C2EE5, taskId=task_1452796302339_0001_1_01_000000, scheduledTime=1452796302576, launchTime=1452796302576
2016-01-14 18:31:42,577 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302339_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:42,576 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:42,577 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:42,577 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5]
2016-01-14 18:31:42,577 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: B07E732BBDF9404B98DC2F69750C2EE5, VertexParallelism: 1, TaskAttemptID:attempt_1452796302339_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C85853AB45884297B4F582A35FE26291, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EA579CA08502432FB871DD843EF39EA1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:42,577 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302339_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:42,578 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:42,578 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796302339_0001_00_000002
2016-01-14 18:31:42,578 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:42,580 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796302339_0001_00_000002
2016-01-14 18:31:42,580 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:42,581 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:42,581 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:42,581 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796302339_0001_00_000002, launchTime=1452796302581
2016-01-14 18:31:42,582 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796302339_0001_00_000002
2016-01-14 18:31:42,582 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796302339_0001_1_01_000000_0] to container: [container_1452796302339_0001_00_000002]
2016-01-14 18:31:42,582 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:42,583 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796302339_0001_00_000002 given task: attempt_1452796302339_0001_1_01_000000_0
2016-01-14 18:31:42,583 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796302339_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796302339_0001_1_01_000000_0
2016-01-14 18:31:42,583 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796302339_0001_1_01_000000_0] started. Is using containerId: [container_1452796302339_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:42,583 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:42,583 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B07E732BBDF9404B98DC2F69750C2EE5, taskAttemptId=attempt_1452796302339_0001_1_01_000000_0, startTime=1452796302583, containerId=container_1452796302339_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796302339_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:42 Starting to run new task attempt: attempt_1452796302339_0001_1_01_000000_0
2016-01-14 18:31:42,583 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302339_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:42,584 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796302339_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:42,584 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302339_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:42,584 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796302339_0001_1_01_000000_0
2016-01-14 18:31:42,584 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:42,584 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796302339_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:42,584 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796302339_0001_00_000001, Exiting
2016-01-14 18:31:42,584 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796302339_0001_00_000001
2016-01-14 18:31:42,584 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:42,585 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:42,585 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796302339_0001_00_000001
2016-01-14 18:31:42,585 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:42,585 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:42,585 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:42,585 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1] every/(1/1) ...yplatform/ALL_UNKNOWN_ALL, VertexName: B07E732BBDF9404B98DC2F69750C2EE5, VertexParallelism: 1, TaskAttemptID:attempt_1452796302339_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C85853AB45884297B4F582A35FE26291, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EA579CA08502432FB871DD843EF39EA1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:42,586 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:42,586 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796302339_0001_1_01_000000_0
2016-01-14 18:31:42,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:42,586 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:42,587 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:42,589 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796302339_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:42,590 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C85853AB45884297B4F582A35FE26291, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:42,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:42,590 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796302339_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:42,591 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796302339_0001_00_000001 completed successfully
2016-01-14 18:31:42,591 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:42,590 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EA579CA08502432FB871DD843EF39EA1, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:42,590 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:42,592 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:42,592 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EA579CA08502432FB871DD843EF39EA1
2016-01-14 18:31:42,592 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:42,592 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: EA579CA08502432FB871DD843EF39EA1
2016-01-14 18:31:42,591 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C85853AB45884297B4F582A35FE26291
2016-01-14 18:31:42,594 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B07E732BBDF9404B98DC2F69750C2EE5, ordinal: 1
2016-01-14 18:31:42,594 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B07E732BBDF9404B98DC2F69750C2EE5, mem on start (mb), free: 299, total: 564, max: 672
2016-01-14 18:31:42,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:42,594 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:42,596 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:42,596 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:42,596 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C85853AB45884297B4F582A35FE26291
2016-01-14 18:31:42,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C85853AB45884297B4F582A35FE26291
2016-01-14 18:31:42,596 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:42,601 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:42,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EA579CA08502432FB871DD843EF39EA1
2016-01-14 18:31:42,602 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:42,602 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:42,603 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:42,603 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:42,603 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:42,604 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:42,604 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:42,604 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EA579CA08502432FB871DD843EF39EA1, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:42,604 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C85853AB45884297B4F582A35FE26291, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:42,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:42,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C85853AB45884297B4F582A35FE26291 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:42,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C85853AB45884297B4F582A35FE26291
2016-01-14 18:31:42,606 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:42,606 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:42,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:42,606 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C85853AB45884297B4F582A35FE26291 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:42,606 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:42,606 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:42,607 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:42,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:42,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C85853AB45884297B4F582A35FE26291
2016-01-14 18:31:42,607 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:42,608 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796302339_0001_1_01_000000_0
2016-01-14 18:31:42,608 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796302339_0001_1_00_000000_0_10273, runDuration: 38120, ]
2016-01-14 18:31:42,610 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796302339_0001_1_00_000000_0_10273] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796302610, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:42,610 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C85853AB45884297B4F582A35FE26291
2016-01-14 18:31:42,610 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:42,610 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C85853AB45884297B4F582A35FE26291] #46 in 1ms
2016-01-14 18:31:42,610 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C85853AB45884297B4F582A35FE26291] #46, status:false, isInterrupted:false
2016-01-14 18:31:42,610 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:42,611 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:42,611 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:42,611 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: BD170B8904434947B2ED9E1C012BCAEE
2016-01-14 18:31:42,611 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:42,612 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:42,612 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:42,612 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C85853AB45884297B4F582A35FE26291
2016-01-14 18:31:42,612 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:42,612 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL"], id: EA579CA08502432FB871DD843EF39EA1
2016-01-14 18:31:42,612 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_ALL"]
2016-01-14 18:31:42,613 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] BD170B8904434947B2ED9E1C012BCAEE, for 1 inputs
2016-01-14 18:31:42,613 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B07E732BBDF9404B98DC2F69750C2EE5, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:42,613 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@18949ad1; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@44401bf2
2016-01-14 18:31:42,617 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:42,617 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:42,617 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796302339_0001_1_01_000000_0
2016-01-14 18:31:42,618 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796302339_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:42,618 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796302339_0001_1_01_000000_0
2016-01-14 18:31:42,618 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B07E732BBDF9404B98DC2F69750C2EE5, mem on close (mb), free: 295, total: 564, max: 672
2016-01-14 18:31:42,619 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C85853AB45884297B4F582A35FE26291
2016-01-14 18:31:42,619 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:42,619 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:42,619 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:42,619 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:42,619 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:42 Completed running task attempt: attempt_1452796302339_0001_1_01_000000_0
2016-01-14 18:31:42,620 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796302339_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:42,621 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796302613, Process_Duration=5, Process_End_Time=1452796302618, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:42,621 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B07E732BBDF9404B98DC2F69750C2EE5, taskAttemptId=attempt_1452796302339_0001_1_01_000000_0, startTime=1452796302583, finishTime=1452796302621, timeTaken=38, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302613, Process_Duration=5, Process_End_Time=1452796302618, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:42,622 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:42,621 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:42,623 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:42,623 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796302339_0001_00_000002
2016-01-14 18:31:42,623 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302339_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:42,623 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796302339_0001_1_01_000000_0
2016-01-14 18:31:42,623 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:42,623 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:TASK_FINISHED]: vertexName=B07E732BBDF9404B98DC2F69750C2EE5, taskId=task_1452796302339_0001_1_01_000000, startTime=1452796302583, finishTime=1452796302623, timeTaken=40, status=SUCCEEDED, successfulAttemptID=attempt_1452796302339_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302613, Process_Duration=5, Process_End_Time=1452796302618, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:42,624 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302339_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:42,624 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5] : 1
2016-01-14 18:31:42,624 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5]
2016-01-14 18:31:42,624 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:42,624 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:VERTEX_FINISHED]: vertexName=B07E732BBDF9404B98DC2F69750C2EE5, vertexId=vertex_1452796302339_0001_1_01, initRequestedTime=1452796302500, initedTime=1452796302508, startRequestedTime=1452796302515, startedTime=1452796302515, finishTime=1452796302624, timeTaken=109, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=238, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796302613, Process_Duration=5, Process_End_Time=1452796302618, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796302583, firstTasksToStart=[ task_1452796302339_0001_1_01_000000 ], lastTaskFinishTime=1452796302621, lastTasksToFinish=[ task_1452796302339_0001_1_01_000000 ], minTaskDuration=38, maxTaskDuration=38, avgTaskDuration=38.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796302339_0001_1_01_000000 ], longestDurationTasks=[ task_1452796302339_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:42,624 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:42,624 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:42,625 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796302339_0001_00_000002, stoppedTime=1452796302625, exitStatus=0
2016-01-14 18:31:42,625 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:42,625 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:42,625 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:42,625 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796302339_0001_1
2016-01-14 18:31:42,625 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796302339_0001_1
2016-01-14 18:31:42,625 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: EA579CA08502432FB871DD843EF39EA1 for vertex: vertex_1452796302339_0001_1_01 [B07E732BBDF9404B98DC2F69750C2EE5]
2016-01-14 18:31:42,625 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:42,632 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796302339_0001_1_00 [C85853AB45884297B4F582A35FE26291]
2016-01-14 18:31:42,633 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796302339_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:42,633 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796302339_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:42,633 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796302339_0001_00_000002, Exiting
2016-01-14 18:31:42,633 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796302339_0001_00_000002
2016-01-14 18:31:42,633 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796302339_0001_00_000002
2016-01-14 18:31:42,634 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796302339_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:42,634 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796302339_0001_00_000002 completed successfully
2016-01-14 18:31:42,634 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796302339_0001_1, queueSize=0
2016-01-14 18:31:42,634 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:42,634 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302339_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796302339_0001_1, startTime=1452796302498, finishTime=1452796302632, timeTaken=134, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=1091, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1182793728, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592605179, Process_Duration=8, Process_End_Time=2905592605187, Read_Duration=3, Tuples_Read=40, Tuples_Written=40, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:42,634 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796302339_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:42,634 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796302339_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:42,634 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302339_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:42 Completed Dag: dag_1452796302339_0001_1
2016-01-14 18:31:42,635 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796302339_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:42,635 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796302339_0001_1_post
2016-01-14 18:31:42,635 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:42,635 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:42,635 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796302339_0001_00_000002
2016-01-14 18:31:42,640 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_UNKNOWN_ALL, applicationId=application_1452796302339_0001
2016-01-14 18:31:42,642 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:42,642 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:42,642 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:42,642 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:42,642 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:42,643 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:42,643 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:42,643 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:42,643 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:42,643 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:42,643 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:42,643 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:42,644 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:42,647 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E28FFA864DD94FBC8ADD1FBAA401CE4D/A970773D942C4A39BAC40A04324CADF1/.tez/application_1452796302339_0001
2016-01-14 18:31:42,648 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.328
2016-01-14 18:31:42,715 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_ALL
2016-01-14 18:31:42,737 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:42,738 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:42,772 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.010
2016-01-14 18:31:42,776 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:42,776 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:42,777 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 18:31:42,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:42,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:42,778 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:42,792 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:42,797 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE
2016-01-14 18:31:42,841 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:42,841 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:42,841 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE"]
2016-01-14 18:31:42,842 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:42,842 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:42,842 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:42,842 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 18:31:42,842 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:42,854 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:42,858 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:42,859 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:42,859 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:42,868 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA/.tez/application_1452796302858_0001 doesn't exist and is created
2016-01-14 18:31:42,880 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:42,881 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:42,881 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA/.tez/application_1452796302858_0001
2016-01-14 18:31:42,882 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796302858_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:42,898 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:42,898 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:42,899 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:42,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:42,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA/.tez/application_1452796302858_0001/localmode-log-dir
2016-01-14 18:31:42,900 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA/.tez/application_1452796302858_0001/localmode-log-dir/history.txt.appattempt_1452796302858_0001_000000, maxErrors=10
2016-01-14 18:31:42,900 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:42,900 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796302858_0001_000000, appSubmitTime=1452796302882, launchTime=1452796302882
2016-01-14 18:31:42,903 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:42,903 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:42,903 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:42,904 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:42,906 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:33530
2016-01-14 18:31:42,907 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:42,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796302858_0001_000000, startTime=1452796302909
2016-01-14 18:31:42,909 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:42,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:42,981 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:42,981 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:42,981 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1452796302858_0001, dagName=[495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 18:31:42,996 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 18:31:42,998 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796302858_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA/.tez/application_1452796302858_0001/localmode-log-dir/dag_1452796302858_0001_1.dot
2016-01-14 18:31:42,998 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA/.tez/application_1452796302858_0001/localmode-log-dir/dag_1452796302858_0001_1-tez-dag.pb.txt
2016-01-14 18:31:43,006 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796302858_0001_1
2016-01-14 18:31:43,006 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,006 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 18:31:43 Running Dag: dag_1452796302858_0001_1
2016-01-14 18:31:43,015 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:43,015 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796302858_0001_1, submitTime=1452796302996
2016-01-14 18:31:43,016 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CC5E558482C744938D2620451789394B
2016-01-14 18:31:43,016 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:43,016 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:43,016 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796302858_0001_1, initTime=1452796303015
2016-01-14 18:31:43,016 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796302858_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:43,017 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:DAG_STARTED]: dagID=dag_1452796302858_0001_1, startTime=1452796303016
2016-01-14 18:31:43,017 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:43,017 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796302858_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:43,017 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:43,017 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F0FA68B2DBF84216B264C1D65F187C1C : {4F26DFAA972D4B99830F3192F2706124={InputName=4F26DFAA972D4B99830F3192F2706124}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:43,017 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1452796302858_0001, dagName=[495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 18:31:43,017 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 4F26DFAA972D4B99830F3192F2706124, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:43,018 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C]
2016-01-14 18:31:43,018 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C]
2016-01-14 18:31:43,018 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C]
2016-01-14 18:31:43,019 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C]. Starting root input initializers: 1
2016-01-14 18:31:43,019 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:43,019 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B]
2016-01-14 18:31:43,022 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:43,023 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B]
2016-01-14 18:31:43,023 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B]
2016-01-14 18:31:43,023 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B]
2016-01-14 18:31:43,023 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=987A6BEFF2E54BB6A5C02833DBE2B1A4, vertexId=vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:43,024 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=987A6BEFF2E54BB6A5C02833DBE2B1A4, vertexId=vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B]
2016-01-14 18:31:43,026 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=987A6BEFF2E54BB6A5C02833DBE2B1A4, vertexName=CC5E558482C744938D2620451789394B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:43,026 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=987A6BEFF2E54BB6A5C02833DBE2B1A4, vertexId=vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B]
2016-01-14 18:31:43,027 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:43,027 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CC5E558482C744938D2620451789394B, vertexId=vertex_1452796302858_0001_1_01, initRequestedTime=1452796303019, initedTime=1452796303027, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:43,027 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:43,027 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 4F26DFAA972D4B99830F3192F2706124 on vertex vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C]
2016-01-14 18:31:43,030 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:43,030 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 4F26DFAA972D4B99830F3192F2706124 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:43,031 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:43,031 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:43,032 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:43,032 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 4F26DFAA972D4B99830F3192F2706124 on vertex vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C]
2016-01-14 18:31:43,032 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:43,033 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {4F26DFAA972D4B99830F3192F2706124=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:43,033 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C] parallelism set to 1
2016-01-14 18:31:43,033 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:43,033 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796302858_0001_1_00, eventCount=1
2016-01-14 18:31:43,034 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:43,034 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F0FA68B2DBF84216B264C1D65F187C1C, vertexId=vertex_1452796302858_0001_1_00, initRequestedTime=1452796303017, initedTime=1452796303033, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:43,034 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C] with distanceFromRoot: 0
2016-01-14 18:31:43,034 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:43,034 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F0FA68B2DBF84216B264C1D65F187C1C
2016-01-14 18:31:43,034 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796302858_0001, with dag id: dag_1452796302858_0001_1
2016-01-14 18:31:43,034 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C]
2016-01-14 18:31:43,035 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796302858_0001_1_00, startRequestedTime=1452796303034, startedTime=1452796303034
2016-01-14 18:31:43,035 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:43,035 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:43,036 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:43,036 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:TASK_STARTED]: vertexName=F0FA68B2DBF84216B264C1D65F187C1C, taskId=task_1452796302858_0001_1_00_000000, scheduledTime=1452796303036, launchTime=1452796303036
2016-01-14 18:31:43,036 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302858_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:43,036 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796302858_0001_1_00 for vertex: vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] numStartedSources: 1 numSources: 1
2016-01-14 18:31:43,037 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] with distanceFromRoot: 1
2016-01-14 18:31:43,037 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C]
2016-01-14 18:31:43,037 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F0FA68B2DBF84216B264C1D65F187C1C in vertex: CC5E558482C744938D2620451789394B
2016-01-14 18:31:43,037 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CC5E558482C744938D2620451789394B with 1 source tasks and 1 pending tasks
2016-01-14 18:31:43,038 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796302858_0001_1_01, startRequestedTime=1452796303037, startedTime=1452796303037
2016-01-14 18:31:43,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:43,038 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:43,038 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: F0FA68B2DBF84216B264C1D65F187C1C, VertexParallelism: 1, TaskAttemptID:attempt_1452796302858_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4F26DFAA972D4B99830F3192F2706124, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CC5E558482C744938D2620451789394B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:43,039 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302858_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:43,039 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:43,040 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:43,041 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:43,041 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:43,041 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796302858_0001_00_000001
2016-01-14 18:31:43,041 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:43,042 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796302858_0001_00_000001
2016-01-14 18:31:43,043 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:43,043 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:43,043 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:43,043 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796302858_0001_00_000001, launchTime=1452796303043
2016-01-14 18:31:43,044 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796302858_0001_00_000001
2016-01-14 18:31:43,044 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796302858_0001_1_00_000000_0] to container: [container_1452796302858_0001_00_000001]
2016-01-14 18:31:43,044 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:43,044 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796302858_0001_1_00_000000_0] started. Is using containerId: [container_1452796302858_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:43,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:43,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F0FA68B2DBF84216B264C1D65F187C1C, taskAttemptId=attempt_1452796302858_0001_1_00_000000_0, startTime=1452796303044, containerId=container_1452796302858_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796302858_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:43,045 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302858_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:43,045 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302858_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:43,045 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796302858_0001_00_000001 given task: attempt_1452796302858_0001_1_00_000000_0
2016-01-14 18:31:43,045 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796302858_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796302858_0001_1_00_000000_0
2016-01-14 18:31:43 Starting to run new task attempt: attempt_1452796302858_0001_1_00_000000_0
2016-01-14 18:31:43,046 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796302858_0001_1_00_000000_0
2016-01-14 18:31:43,046 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,046 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:43,046 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:43,048 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:43,048 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:43,048 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:43,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: F0FA68B2DBF84216B264C1D65F187C1C, VertexParallelism: 1, TaskAttemptID:attempt_1452796302858_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=4F26DFAA972D4B99830F3192F2706124, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CC5E558482C744938D2620451789394B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:43,049 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:43,049 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796302858_0001_1_00_000000_0
2016-01-14 18:31:43,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:43,049 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,049 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796302858_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:43,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CC5E558482C744938D2620451789394B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:43,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:43,050 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CC5E558482C744938D2620451789394B
2016-01-14 18:31:43,052 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:43,052 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CC5E558482C744938D2620451789394B
2016-01-14 18:31:43,052 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CC5E558482C744938D2620451789394B
2016-01-14 18:31:43,052 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:43,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=4F26DFAA972D4B99830F3192F2706124, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:43,054 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:43,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 4F26DFAA972D4B99830F3192F2706124
2016-01-14 18:31:43,055 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 4F26DFAA972D4B99830F3192F2706124
2016-01-14 18:31:43,056 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:43,056 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 4F26DFAA972D4B99830F3192F2706124
2016-01-14 18:31:43,057 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 4F26DFAA972D4B99830F3192F2706124
2016-01-14 18:31:43,057 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F0FA68B2DBF84216B264C1D65F187C1C, ordinal: 0
2016-01-14 18:31:43,057 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F0FA68B2DBF84216B264C1D65F187C1C, mem on start (mb), free: 187, total: 564, max: 672
2016-01-14 18:31:43,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:43,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:43,058 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:43,058 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:43,058 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:43,058 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:43,058 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:43,058 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:43,059 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:43,059 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CC5E558482C744938D2620451789394B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:43,059 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 4F26DFAA972D4B99830F3192F2706124, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:43,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:43,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 4F26DFAA972D4B99830F3192F2706124 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:43,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:43,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:43,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 4F26DFAA972D4B99830F3192F2706124
2016-01-14 18:31:43,060 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:43,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 4F26DFAA972D4B99830F3192F2706124
2016-01-14 18:31:43,060 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:43,061 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796302858_0001_1_00_000000_0
2016-01-14 18:31:43,061 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:43,061 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@62b8054d
2016-01-14 18:31:43,062 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@62b8054d
2016-01-14 18:31:43,062 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:43,062 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:43,063 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:43,064 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 4F26DFAA972D4B99830F3192F2706124
2016-01-14 18:31:43,064 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 0F6030D99C074F65951F86FDD2EDC0C2
2016-01-14 18:31:43,064 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 0F6030D99C074F65951F86FDD2EDC0C2
2016-01-14 18:31:43,066 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@49aa3fe4; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@1e08602f; comparator=cascading.tuple.hadoop.util.TupleComparator@37e2bd10; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:43,066 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:43,066 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:43,066 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:43,092 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:43,092 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:43,092 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:43,092 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:43,093 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:43,093 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:43,093 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:43,093 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F0FA68B2DBF84216B264C1D65F187C1C, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:43,095 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796302858_0001_1_00_000000_0
2016-01-14 18:31:43,096 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F0FA68B2DBF84216B264C1D65F187C1C, mem on close (mb), free: 297, total: 560, max: 672
2016-01-14 18:31:43,096 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:43,096 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:43,096 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:43,096 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:43,097 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:43 Completed running task attempt: attempt_1452796302858_0001_1_00_000000_0
2016-01-14 18:31:43,100 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796302858_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:43,100 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796302858_0001_1_00, eventCount=1
2016-01-14 18:31:43,100 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F0FA68B2DBF84216B264C1D65F187C1C, taskAttemptId=attempt_1452796302858_0001_1_00_000000_0, startTime=1452796303044, finishTime=1452796303100, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=587202560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796303094, Process_Duration=1, Process_End_Time=1452796303095, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:43,101 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302858_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:43,101 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796302858_0001_1_00_000000_0
2016-01-14 18:31:43,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:43,101 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:43,101 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:TASK_FINISHED]: vertexName=F0FA68B2DBF84216B264C1D65F187C1C, taskId=task_1452796302858_0001_1_00_000000, startTime=1452796303044, finishTime=1452796303101, timeTaken=57, status=SUCCEEDED, successfulAttemptID=attempt_1452796302858_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=587202560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796303094, Process_Duration=1, Process_End_Time=1452796303095, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:43,101 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:43,101 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302858_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:43,101 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C] : 1
2016-01-14 18:31:43,101 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C]
2016-01-14 18:31:43,102 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:43,102 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:VERTEX_FINISHED]: vertexName=F0FA68B2DBF84216B264C1D65F187C1C, vertexId=vertex_1452796302858_0001_1_00, initRequestedTime=1452796303017, initedTime=1452796303033, startRequestedTime=1452796303034, startedTime=1452796303034, finishTime=1452796303102, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=587202560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796303094, Process_Duration=1, Process_End_Time=1452796303095, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796303044, firstTasksToStart=[ task_1452796302858_0001_1_00_000000 ], lastTaskFinishTime=1452796303100, lastTasksToFinish=[ task_1452796302858_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796302858_0001_1_00_000000 ], longestDurationTasks=[ task_1452796302858_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:43,102 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:43,102 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:43,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=587202560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796303094, Process_Duration=1, Process_End_Time=1452796303095, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:43,103 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] attempt: attempt_1452796302858_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:43,103 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CC5E558482C744938D2620451789394B
2016-01-14 18:31:43,103 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B]
2016-01-14 18:31:43,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796302858_0001_00_000001, stoppedTime=1452796303103, exitStatus=0
2016-01-14 18:31:43,103 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:43,103 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:43,103 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:43,104 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:TASK_STARTED]: vertexName=CC5E558482C744938D2620451789394B, taskId=task_1452796302858_0001_1_01_000000, scheduledTime=1452796303103, launchTime=1452796303103
2016-01-14 18:31:43,104 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:43,103 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:43,104 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,104 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302858_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:43,104 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796302858_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:43,104 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B]
2016-01-14 18:31:43,104 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:43,104 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: CC5E558482C744938D2620451789394B, VertexParallelism: 1, TaskAttemptID:attempt_1452796302858_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F0FA68B2DBF84216B264C1D65F187C1C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=987A6BEFF2E54BB6A5C02833DBE2B1A4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:43,104 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302858_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:43,104 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796302858_0001_00_000001
2016-01-14 18:31:43,105 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796302858_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:43,105 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796302858_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:43,105 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796302858_0001_00_000001, Exiting
2016-01-14 18:31:43,105 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796302858_0001_00_000001
2016-01-14 18:31:43,105 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796302858_0001_00_000001
2016-01-14 18:31:43,107 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:43,108 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796302858_0001_00_000002
2016-01-14 18:31:43,108 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796302858_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:43,108 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796302858_0001_00_000001 completed successfully
2016-01-14 18:31:43,108 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:43,109 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:43,110 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796302858_0001_00_000002
2016-01-14 18:31:43,110 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:43,112 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:43,112 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:43,112 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796302858_0001_00_000002, launchTime=1452796303112
2016-01-14 18:31:43,113 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796302858_0001_00_000002
2016-01-14 18:31:43,114 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796302858_0001_1_01_000000_0] to container: [container_1452796302858_0001_00_000002]
2016-01-14 18:31:43,114 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:43,115 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796302858_0001_00_000002 given task: attempt_1452796302858_0001_1_01_000000_0
2016-01-14 18:31:43,115 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796302858_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796302858_0001_1_01_000000_0
2016-01-14 18:31:43,115 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796302858_0001_1_01_000000_0] started. Is using containerId: [container_1452796302858_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:43,115 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CC5E558482C744938D2620451789394B, taskAttemptId=attempt_1452796302858_0001_1_01_000000_0, startTime=1452796303115, containerId=container_1452796302858_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796302858_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:43,115 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302858_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:43,115 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302858_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:43,115 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:43 Starting to run new task attempt: attempt_1452796302858_0001_1_01_000000_0
2016-01-14 18:31:43,115 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796302858_0001_1_01_000000_0
2016-01-14 18:31:43,116 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,116 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:43,116 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:43,116 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:43,116 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:43,117 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:43,117 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA] every/(1/1) ...tform/ALL_UNKNOWN_REPLACE, VertexName: CC5E558482C744938D2620451789394B, VertexParallelism: 1, TaskAttemptID:attempt_1452796302858_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F0FA68B2DBF84216B264C1D65F187C1C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=987A6BEFF2E54BB6A5C02833DBE2B1A4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:43,117 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:43,118 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796302858_0001_1_01_000000_0
2016-01-14 18:31:43,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:43,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,118 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796302858_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:43,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,118 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F0FA68B2DBF84216B264C1D65F187C1C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:43,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:43,121 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F0FA68B2DBF84216B264C1D65F187C1C
2016-01-14 18:31:43,122 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:43,123 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:43,123 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:43,123 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F0FA68B2DBF84216B264C1D65F187C1C
2016-01-14 18:31:43,123 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F0FA68B2DBF84216B264C1D65F187C1C
2016-01-14 18:31:43,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=987A6BEFF2E54BB6A5C02833DBE2B1A4, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:43,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:43,124 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 987A6BEFF2E54BB6A5C02833DBE2B1A4
2016-01-14 18:31:43,124 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:43,125 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 987A6BEFF2E54BB6A5C02833DBE2B1A4
2016-01-14 18:31:43,127 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CC5E558482C744938D2620451789394B, ordinal: 1
2016-01-14 18:31:43,127 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CC5E558482C744938D2620451789394B, mem on start (mb), free: 291, total: 560, max: 672
2016-01-14 18:31:43,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:43,127 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:43,132 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:43,132 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 987A6BEFF2E54BB6A5C02833DBE2B1A4
2016-01-14 18:31:43,133 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:43,133 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:43,133 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:43,133 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:43,133 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:43,133 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:43,134 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:43,134 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F0FA68B2DBF84216B264C1D65F187C1C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:43,134 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 987A6BEFF2E54BB6A5C02833DBE2B1A4, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:43,134 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:43,134 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F0FA68B2DBF84216B264C1D65F187C1C
2016-01-14 18:31:43,135 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:43,135 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:43,135 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F0FA68B2DBF84216B264C1D65F187C1C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:43,135 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:43,135 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:43,135 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:43,135 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F0FA68B2DBF84216B264C1D65F187C1C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:43,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:43,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:43,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F0FA68B2DBF84216B264C1D65F187C1C
2016-01-14 18:31:43,136 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:43,137 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796302858_0001_1_01_000000_0
2016-01-14 18:31:43,137 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796302858_0001_1_00_000000_0_10278, runDuration: 47889, ]
2016-01-14 18:31:43,139 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796302858_0001_1_00_000000_0_10278] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796303139, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:43,139 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F0FA68B2DBF84216B264C1D65F187C1C
2016-01-14 18:31:43,139 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:43,139 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F0FA68B2DBF84216B264C1D65F187C1C] #47 in 1ms
2016-01-14 18:31:43,139 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F0FA68B2DBF84216B264C1D65F187C1C] #47, status:false, isInterrupted:false
2016-01-14 18:31:43,140 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:43,140 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:43,141 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 0F6030D99C074F65951F86FDD2EDC0C2
2016-01-14 18:31:43,141 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:43,141 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:43,142 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:43,142 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE"], id: 987A6BEFF2E54BB6A5C02833DBE2B1A4
2016-01-14 18:31:43,142 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:43,142 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F0FA68B2DBF84216B264C1D65F187C1C
2016-01-14 18:31:43,142 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:43,142 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_REPLACE"]
2016-01-14 18:31:43,143 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 0F6030D99C074F65951F86FDD2EDC0C2, for 1 inputs
2016-01-14 18:31:43,143 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CC5E558482C744938D2620451789394B, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:43,143 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@46449ea4; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@430b4506
2016-01-14 18:31:43,144 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:43,144 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:43,144 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:43,145 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:43,145 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796302858_0001_1_01_000000_0
2016-01-14 18:31:43,145 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796302858_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:43,146 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796302858_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:43,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=587202560, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796303143, Process_Duration=3, Process_End_Time=1452796303146, Read_Duration=1, Tuples_Read=1]]
2016-01-14 18:31:43,147 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CC5E558482C744938D2620451789394B, taskAttemptId=attempt_1452796302858_0001_1_01_000000_0, startTime=1452796303115, finishTime=1452796303147, timeTaken=32, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=587202560, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796303143, Process_Duration=3, Process_End_Time=1452796303146, Read_Duration=1, Tuples_Read=1
2016-01-14 18:31:43,147 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302858_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:43,147 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CC5E558482C744938D2620451789394B, mem on close (mb), free: 288, total: 560, max: 672
2016-01-14 18:31:43,147 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796302858_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:43,148 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:TASK_FINISHED]: vertexName=CC5E558482C744938D2620451789394B, taskId=task_1452796302858_0001_1_01_000000, startTime=1452796303115, finishTime=1452796303148, timeTaken=33, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:43,148 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796302858_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:43,148 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] : 1
2016-01-14 18:31:43,148 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] because task failed: task_1452796302858_0001_1_01_000000
2016-01-14 18:31:43,147 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:43,148 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:43,148 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:43,148 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:43,149 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B]
2016-01-14 18:31:43,149 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:43,147 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=CC5E558482C744938D2620451789394B, index=1
2016-01-14 18:31:43,149 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=987A6BEFF2E54BB6A5C02833DBE2B1A4, vertexId=vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B]
2016-01-14 18:31:43,149 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F0FA68B2DBF84216B264C1D65F187C1C
2016-01-14 18:31:43,149 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:43,149 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:43,150 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:43,150 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:43,150 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:43,150 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:43,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=CC5E558482C744938D2620451789394B, sourceVertex=F0FA68B2DBF84216B264C1D65F187C1C
2016-01-14 18:31:43,150 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:VERTEX_FINISHED]: vertexName=CC5E558482C744938D2620451789394B, vertexId=vertex_1452796302858_0001_1_01, initRequestedTime=1452796303019, initedTime=1452796303027, startRequestedTime=1452796303037, startedTime=1452796303037, finishTime=1452796303150, timeTaken=113, status=FAILED, diagnostics=Task failed, taskId=task_1452796302858_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796303115, firstTasksToStart=[ task_1452796302858_0001_1_01_000000 ], lastTaskFinishTime=1452796303147, lastTasksToFinish=[ task_1452796302858_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:43,151 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:43,151 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796302858_0001_00_000002, stoppedTime=1452796303151, exitStatus=0
2016-01-14 18:31:43,151 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:43,151 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796302858_0001_1_01_000000_0 failed: true container: container_1452796302858_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:43,150 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=CC5E558482C744938D2620451789394B, sourceVertex=987A6BEFF2E54BB6A5C02833DBE2B1A4
2016-01-14 18:31:43,151 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:43,151 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:43,152 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:43,152 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796302858_0001_1
2016-01-14 18:31:43,152 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 987A6BEFF2E54BB6A5C02833DBE2B1A4 for vertex: vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B]
2016-01-14 18:31:43,152 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796302858_0001_1_00 [F0FA68B2DBF84216B264C1D65F187C1C]
2016-01-14 18:31:43,152 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796302858_0001_00_000002
2016-01-14 18:31:43,152 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796302858_0001_00_000002
2016-01-14 18:31:43,152 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796302858_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:43,153 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796302858_0001_00_000002 completed but with errors
2016-01-14 18:31:43,154 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796302858_0001_1, queueSize=0
2016-01-14 18:31:43,154 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:43,155 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796302858_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796302858_0001_1, startTime=1452796303016, finishTime=1452796303152, timeTaken=136, status=FAILED, diagnostics=Vertex failed, vertexName=CC5E558482C744938D2620451789394B, vertexId=vertex_1452796302858_0001_1_01, diagnostics=[Task failed, taskId=task_1452796302858_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796302858_0001_1_01 [CC5E558482C744938D2620451789394B] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=587202560, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796303094, Process_Duration=1, Process_End_Time=1452796303095, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:43,155 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796302858_0001_1 finished with state: FAILED
2016-01-14 18:31:43,155 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796302858_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:43,155 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796302858_0001_00_000002
2016-01-14 18:31:43,155 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796302858_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:43,155 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796302858_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:43 Completed Dag: dag_1452796302858_0001_1
2016-01-14 18:31:43,155 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796302858_0001_1, dagState=FAILED
2016-01-14 18:31:43,155 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796302858_0001_1_post
2016-01-14 18:31:43,155 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,156 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:43,156 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796302858_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:43,160 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...tform/ALL_UNKNOWN_REPLACE, applicationId=application_1452796302858_0001
2016-01-14 18:31:43,162 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:43,162 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:43,162 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:43,162 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:43,162 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:43,163 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:43,163 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:43,163 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:43,163 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:43,164 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:43,164 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:43,164 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:43,164 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:43,168 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/495D0E5C3F6548468466EF4B4BA63A9F/FB5F7B5C506240B08F763F81316D67EA/.tez/application_1452796302858_0001
2016-01-14 18:31:43,168 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:43,168 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...tform/ALL_UNKNOWN_REPLACE
2016-01-14 18:31:43,169 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:43,169 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.328
2016-01-14 18:31:43,169 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_REPLACE
2016-01-14 18:31:43,192 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:43,192 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:43,225 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.008
2016-01-14 18:31:43,230 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.013
2016-01-14 18:31:43,230 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:43,230 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:43,231 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:43,245 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:43,251 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP
2016-01-14 18:31:43,296 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:43,296 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:43,296 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP"]
2016-01-14 18:31:43,296 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:43,296 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:43,297 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:43,297 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/ALL_UNKNOWN_SWAP
2016-01-14 18:31:43,297 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:43,309 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:43,313 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:43,313 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:43,313 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:43,322 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D/.tez/application_1452796303313_0001 doesn't exist and is created
2016-01-14 18:31:43,336 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:43,336 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:43,337 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D/.tez/application_1452796303313_0001
2016-01-14 18:31:43,338 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796303313_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:43,346 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:43,346 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:43,350 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:43,351 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:43,352 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:43,352 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:43,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D/.tez/application_1452796303313_0001/localmode-log-dir
2016-01-14 18:31:43,352 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D/.tez/application_1452796303313_0001/localmode-log-dir/history.txt.appattempt_1452796303313_0001_000000, maxErrors=10
2016-01-14 18:31:43,352 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:43,353 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796303313_0001_000000, appSubmitTime=1452796303338, launchTime=1452796303338
2016-01-14 18:31:43,358 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:43,358 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:43,359 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:43,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:43,361 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:55943
2016-01-14 18:31:43,362 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:43,364 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796303313_0001_000000, startTime=1452796303364
2016-01-14 18:31:43,364 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:43,364 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:43,437 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:43,437 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:43,437 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1452796303313_0001, dagName=[E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-01-14 18:31:43,452 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-01-14 18:31:43,453 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796303313_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D/.tez/application_1452796303313_0001/localmode-log-dir/dag_1452796303313_0001_1.dot
2016-01-14 18:31:43,454 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D/.tez/application_1452796303313_0001/localmode-log-dir/dag_1452796303313_0001_1-tez-dag.pb.txt
2016-01-14 18:31:43,461 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796303313_0001_1
2016-01-14 18:31:43,461 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,461 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-01-14 18:31:43 Running Dag: dag_1452796303313_0001_1
2016-01-14 18:31:43,470 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796303313_0001_1, submitTime=1452796303452
2016-01-14 18:31:43,470 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:43,473 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BD604E693AEA4BC0B297A126BCAE3657
2016-01-14 18:31:43,473 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:43,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796303313_0001_1, initTime=1452796303473
2016-01-14 18:31:43,474 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796303313_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:43,474 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:DAG_STARTED]: dagID=dag_1452796303313_0001_1, startTime=1452796303474
2016-01-14 18:31:43,474 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:43,474 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:43,475 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1452796303313_0001, dagName=[E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D] every/(1/1) ...platform/ALL_UNKNOWN_SWAP
2016-01-14 18:31:43,475 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:43,475 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796303313_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:43,475 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D8EB5CBB634C4D438CF09C7BC8E97D9A : {CA0981B1EB144611946B93B68A71333C={InputName=CA0981B1EB144611946B93B68A71333C}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:43,475 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: CA0981B1EB144611946B93B68A71333C, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:43,475 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A]
2016-01-14 18:31:43,475 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A]
2016-01-14 18:31:43,475 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A]
2016-01-14 18:31:43,476 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A]. Starting root input initializers: 1
2016-01-14 18:31:43,476 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:43,476 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657]
2016-01-14 18:31:43,477 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: CA0981B1EB144611946B93B68A71333C on vertex vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A]
2016-01-14 18:31:43,479 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:43,479 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:43,479 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657]
2016-01-14 18:31:43,480 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657]
2016-01-14 18:31:43,480 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657]
2016-01-14 18:31:43,480 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=EC1960F079564C6AAC5C20B085C5BAEB, vertexId=vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:43,480 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=EC1960F079564C6AAC5C20B085C5BAEB, vertexId=vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657]
2016-01-14 18:31:43,479 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input CA0981B1EB144611946B93B68A71333C asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:43,481 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:43,481 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:43,483 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=EC1960F079564C6AAC5C20B085C5BAEB, vertexName=BD604E693AEA4BC0B297A126BCAE3657, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:43,483 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=EC1960F079564C6AAC5C20B085C5BAEB, vertexId=vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657]
2016-01-14 18:31:43,483 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:43,484 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: CA0981B1EB144611946B93B68A71333C on vertex vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A]
2016-01-14 18:31:43,484 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:43,484 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BD604E693AEA4BC0B297A126BCAE3657, vertexId=vertex_1452796303313_0001_1_01, initRequestedTime=1452796303476, initedTime=1452796303484, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:43,484 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:43,484 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:43,485 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {CA0981B1EB144611946B93B68A71333C=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:43,485 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A] parallelism set to 1
2016-01-14 18:31:43,485 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:43,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796303313_0001_1_00, eventCount=1
2016-01-14 18:31:43,486 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:43,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D8EB5CBB634C4D438CF09C7BC8E97D9A, vertexId=vertex_1452796303313_0001_1_00, initRequestedTime=1452796303475, initedTime=1452796303486, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:43,486 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A] with distanceFromRoot: 0
2016-01-14 18:31:43,486 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:43,487 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D8EB5CBB634C4D438CF09C7BC8E97D9A
2016-01-14 18:31:43,487 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A]
2016-01-14 18:31:43,487 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:43,487 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796303313_0001_1_00, startRequestedTime=1452796303487, startedTime=1452796303487
2016-01-14 18:31:43,488 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:43,488 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:43,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:TASK_STARTED]: vertexName=D8EB5CBB634C4D438CF09C7BC8E97D9A, taskId=task_1452796303313_0001_1_00_000000, scheduledTime=1452796303488, launchTime=1452796303488
2016-01-14 18:31:43,488 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303313_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:43,489 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796303313_0001_1_00 for vertex: vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657] numStartedSources: 1 numSources: 1
2016-01-14 18:31:43,489 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657] with distanceFromRoot: 1
2016-01-14 18:31:43,489 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A]
2016-01-14 18:31:43,489 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D8EB5CBB634C4D438CF09C7BC8E97D9A in vertex: BD604E693AEA4BC0B297A126BCAE3657
2016-01-14 18:31:43,490 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BD604E693AEA4BC0B297A126BCAE3657 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:43,490 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:43,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796303313_0001_1_01, startRequestedTime=1452796303489, startedTime=1452796303489
2016-01-14 18:31:43,490 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:43,491 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: D8EB5CBB634C4D438CF09C7BC8E97D9A, VertexParallelism: 1, TaskAttemptID:attempt_1452796303313_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CA0981B1EB144611946B93B68A71333C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD604E693AEA4BC0B297A126BCAE3657, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:43,491 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303313_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:43,491 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:43,492 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:43,492 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:43,492 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:43,493 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796303313_0001_00_000001
2016-01-14 18:31:43,493 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:43,494 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796303313_0001_00_000001
2016-01-14 18:31:43,494 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:43,495 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:43,495 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:43,495 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796303313_0001_00_000001, launchTime=1452796303494
2016-01-14 18:31:43,496 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796303313_0001_00_000001
2016-01-14 18:31:43,496 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796303313_0001_1_00_000000_0] to container: [container_1452796303313_0001_00_000001]
2016-01-14 18:31:43,496 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:43,497 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796303313_0001_00_000001 given task: attempt_1452796303313_0001_1_00_000000_0
2016-01-14 18:31:43,497 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796303313_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796303313_0001_1_00_000000_0
2016-01-14 18:31:43,497 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796303313_0001_1_00_000000_0] started. Is using containerId: [container_1452796303313_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:43,497 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D8EB5CBB634C4D438CF09C7BC8E97D9A, taskAttemptId=attempt_1452796303313_0001_1_00_000000_0, startTime=1452796303497, containerId=container_1452796303313_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796303313_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:43,497 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303313_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:43 Starting to run new task attempt: attempt_1452796303313_0001_1_00_000000_0
2016-01-14 18:31:43,497 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303313_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:43,497 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:43,497 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796303313_0001_1_00_000000_0
2016-01-14 18:31:43,498 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,498 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796303313_0001, with dag id: dag_1452796303313_0001_1
2016-01-14 18:31:43,498 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:43,498 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:43,498 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:43,498 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:43,498 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:43,498 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: D8EB5CBB634C4D438CF09C7BC8E97D9A, VertexParallelism: 1, TaskAttemptID:attempt_1452796303313_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=CA0981B1EB144611946B93B68A71333C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BD604E693AEA4BC0B297A126BCAE3657, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:43,498 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:43,499 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796303313_0001_1_00_000000_0
2016-01-14 18:31:43,499 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:43,499 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,499 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796303313_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:43,500 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,500 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BD604E693AEA4BC0B297A126BCAE3657, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:43,500 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:43,501 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BD604E693AEA4BC0B297A126BCAE3657
2016-01-14 18:31:43,502 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:43,502 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:43,503 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BD604E693AEA4BC0B297A126BCAE3657
2016-01-14 18:31:43,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BD604E693AEA4BC0B297A126BCAE3657
2016-01-14 18:31:43,503 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=CA0981B1EB144611946B93B68A71333C, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:43,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:43,504 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: CA0981B1EB144611946B93B68A71333C
2016-01-14 18:31:43,504 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: CA0981B1EB144611946B93B68A71333C
2016-01-14 18:31:43,506 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:43,507 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: CA0981B1EB144611946B93B68A71333C
2016-01-14 18:31:43,507 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D8EB5CBB634C4D438CF09C7BC8E97D9A, ordinal: 0
2016-01-14 18:31:43,507 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8EB5CBB634C4D438CF09C7BC8E97D9A, mem on start (mb), free: 199, total: 560, max: 672
2016-01-14 18:31:43,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:43,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:43,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: CA0981B1EB144611946B93B68A71333C
2016-01-14 18:31:43,507 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:43,508 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:43,508 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:43,508 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:43,508 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:43,508 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:43,508 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:43,509 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BD604E693AEA4BC0B297A126BCAE3657, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:43,509 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, CA0981B1EB144611946B93B68A71333C, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:43,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:43,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: CA0981B1EB144611946B93B68A71333C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:43,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:43,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:43,509 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: CA0981B1EB144611946B93B68A71333C
2016-01-14 18:31:43,510 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:43,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: CA0981B1EB144611946B93B68A71333C
2016-01-14 18:31:43,510 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:43,510 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796303313_0001_1_00_000000_0
2016-01-14 18:31:43,511 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:43,511 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@36d1b1a5
2016-01-14 18:31:43,512 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@36d1b1a5
2016-01-14 18:31:43,512 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:43,512 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:43,514 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:43,514 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: CA0981B1EB144611946B93B68A71333C
2016-01-14 18:31:43,514 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 74449CBAEB3140939AF6A4CE529968EC
2016-01-14 18:31:43,515 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 74449CBAEB3140939AF6A4CE529968EC
2016-01-14 18:31:43,515 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@3f1d0980; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@b5ad2df; comparator=cascading.tuple.hadoop.util.TupleComparator@77ca344b; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:43,515 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:43,515 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:43,515 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:43,536 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:43,537 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:43,537 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:43,537 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:43,537 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:43,537 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:43,538 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:43,538 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D8EB5CBB634C4D438CF09C7BC8E97D9A, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:43,540 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796303313_0001_1_00_000000_0
2016-01-14 18:31:43,540 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8EB5CBB634C4D438CF09C7BC8E97D9A, mem on close (mb), free: 298, total: 563, max: 672
2016-01-14 18:31:43,541 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:43,541 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:43,541 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:43,541 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:43,543 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:43 Completed running task attempt: attempt_1452796303313_0001_1_00_000000_0
2016-01-14 18:31:43,545 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796303313_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:43,545 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796303313_0001_1_00, eventCount=1
2016-01-14 18:31:43,546 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796303538, Process_Duration=2, Process_End_Time=1452796303540, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:43,546 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:43,546 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:43,546 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,546 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796303313_0001_00_000001
2016-01-14 18:31:43,546 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D8EB5CBB634C4D438CF09C7BC8E97D9A, taskAttemptId=attempt_1452796303313_0001_1_00_000000_0, startTime=1452796303497, finishTime=1452796303546, timeTaken=49, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796303538, Process_Duration=2, Process_End_Time=1452796303540, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:43,547 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303313_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:43,547 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:43,547 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796303313_0001_1_00_000000_0
2016-01-14 18:31:43,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:43,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:TASK_FINISHED]: vertexName=D8EB5CBB634C4D438CF09C7BC8E97D9A, taskId=task_1452796303313_0001_1_00_000000, startTime=1452796303497, finishTime=1452796303547, timeTaken=50, status=SUCCEEDED, successfulAttemptID=attempt_1452796303313_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796303538, Process_Duration=2, Process_End_Time=1452796303540, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:43,547 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303313_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:43,548 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A] : 1
2016-01-14 18:31:43,548 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A]
2016-01-14 18:31:43,548 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:43,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:VERTEX_FINISHED]: vertexName=D8EB5CBB634C4D438CF09C7BC8E97D9A, vertexId=vertex_1452796303313_0001_1_00, initRequestedTime=1452796303475, initedTime=1452796303486, startRequestedTime=1452796303487, startedTime=1452796303487, finishTime=1452796303548, timeTaken=61, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796303538, Process_Duration=2, Process_End_Time=1452796303540, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796303497, firstTasksToStart=[ task_1452796303313_0001_1_00_000000 ], lastTaskFinishTime=1452796303546, lastTasksToFinish=[ task_1452796303313_0001_1_00_000000 ], minTaskDuration=49, maxTaskDuration=49, avgTaskDuration=49.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796303313_0001_1_00_000000 ], longestDurationTasks=[ task_1452796303313_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:43,548 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:43,548 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:43,548 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796303313_0001_00_000001, stoppedTime=1452796303548, exitStatus=0
2016-01-14 18:31:43,548 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:43,549 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:43,549 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657] attempt: attempt_1452796303313_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:43,549 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BD604E693AEA4BC0B297A126BCAE3657
2016-01-14 18:31:43,549 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657]
2016-01-14 18:31:43,549 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:43,549 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796303313_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:43,549 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:TASK_STARTED]: vertexName=BD604E693AEA4BC0B297A126BCAE3657, taskId=task_1452796303313_0001_1_01_000000, scheduledTime=1452796303549, launchTime=1452796303549
2016-01-14 18:31:43,550 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303313_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:43,550 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657]
2016-01-14 18:31:43,550 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:43,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:43,551 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: BD604E693AEA4BC0B297A126BCAE3657, VertexParallelism: 1, TaskAttemptID:attempt_1452796303313_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D8EB5CBB634C4D438CF09C7BC8E97D9A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EC1960F079564C6AAC5C20B085C5BAEB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:43,551 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303313_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:43,551 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:43,552 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796303313_0001_00_000002
2016-01-14 18:31:43,552 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:43,554 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796303313_0001_00_000002
2016-01-14 18:31:43,554 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:43,554 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:43,555 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:43,555 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796303313_0001_00_000002, launchTime=1452796303555
2016-01-14 18:31:43,555 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796303313_0001_00_000002
2016-01-14 18:31:43,556 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796303313_0001_1_01_000000_0] to container: [container_1452796303313_0001_00_000002]
2016-01-14 18:31:43,556 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:43,556 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796303313_0001_00_000002 given task: attempt_1452796303313_0001_1_01_000000_0
2016-01-14 18:31:43,556 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796303313_0001_1_01_000000_0] started. Is using containerId: [container_1452796303313_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:43,556 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796303313_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796303313_0001_1_01_000000_0
2016-01-14 18:31:43,556 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BD604E693AEA4BC0B297A126BCAE3657, taskAttemptId=attempt_1452796303313_0001_1_01_000000_0, startTime=1452796303556, containerId=container_1452796303313_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796303313_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:43,556 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:43,557 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303313_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:43,557 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796303313_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:43,557 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796303313_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:43 Starting to run new task attempt: attempt_1452796303313_0001_1_01_000000_0
2016-01-14 18:31:43,557 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796303313_0001_1_01_000000_0
2016-01-14 18:31:43,557 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,558 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:43,558 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:43,558 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:43,558 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:43,558 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:43,558 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D] every/(1/1) ...platform/ALL_UNKNOWN_SWAP, VertexName: BD604E693AEA4BC0B297A126BCAE3657, VertexParallelism: 1, TaskAttemptID:attempt_1452796303313_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D8EB5CBB634C4D438CF09C7BC8E97D9A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=EC1960F079564C6AAC5C20B085C5BAEB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:43,558 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:43,557 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303313_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:43,557 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796303313_0001_00_000001, Exiting
2016-01-14 18:31:43,559 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796303313_0001_00_000001
2016-01-14 18:31:43,559 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796303313_0001_00_000001
2016-01-14 18:31:43,559 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796303313_0001_1_01_000000_0
2016-01-14 18:31:43,559 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:43,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,560 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796303313_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:43,560 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796303313_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:43,561 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796303313_0001_00_000001 completed successfully
2016-01-14 18:31:43,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=EC1960F079564C6AAC5C20B085C5BAEB, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:43,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:43,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: EC1960F079564C6AAC5C20B085C5BAEB
2016-01-14 18:31:43,561 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:43,561 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: EC1960F079564C6AAC5C20B085C5BAEB
2016-01-14 18:31:43,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D8EB5CBB634C4D438CF09C7BC8E97D9A, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:43,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,566 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:43,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:43,561 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:43,568 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D8EB5CBB634C4D438CF09C7BC8E97D9A
2016-01-14 18:31:43,570 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:43,570 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:43,570 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D8EB5CBB634C4D438CF09C7BC8E97D9A
2016-01-14 18:31:43,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D8EB5CBB634C4D438CF09C7BC8E97D9A
2016-01-14 18:31:43,571 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:43,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: EC1960F079564C6AAC5C20B085C5BAEB
2016-01-14 18:31:43,571 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BD604E693AEA4BC0B297A126BCAE3657, ordinal: 1
2016-01-14 18:31:43,572 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD604E693AEA4BC0B297A126BCAE3657, mem on start (mb), free: 292, total: 563, max: 672
2016-01-14 18:31:43,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:43,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:43,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:43,572 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:43,572 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:43,572 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:43,572 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:43,572 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:43,572 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:43,572 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:43,573 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, EC1960F079564C6AAC5C20B085C5BAEB, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:43,573 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D8EB5CBB634C4D438CF09C7BC8E97D9A, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:43,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:43,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D8EB5CBB634C4D438CF09C7BC8E97D9A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:43,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:43,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:43,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D8EB5CBB634C4D438CF09C7BC8E97D9A
2016-01-14 18:31:43,573 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:43,573 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:43,574 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D8EB5CBB634C4D438CF09C7BC8E97D9A with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:43,574 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:43,574 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:43,574 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:43,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D8EB5CBB634C4D438CF09C7BC8E97D9A
2016-01-14 18:31:43,575 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:43,576 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796303313_0001_1_01_000000_0
2016-01-14 18:31:43,578 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796303313_0001_1_00_000000_0_10284, runDuration: 42886, ]
2016-01-14 18:31:43,578 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:43,579 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 74449CBAEB3140939AF6A4CE529968EC
2016-01-14 18:31:43,579 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP"], id: EC1960F079564C6AAC5C20B085C5BAEB
2016-01-14 18:31:43,579 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_UNKNOWN_SWAP"]
2016-01-14 18:31:43,579 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 74449CBAEB3140939AF6A4CE529968EC, for 1 inputs
2016-01-14 18:31:43,580 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796303313_0001_1_00_000000_0_10284] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796303580, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:43,581 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D8EB5CBB634C4D438CF09C7BC8E97D9A
2016-01-14 18:31:43,581 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:43,581 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D8EB5CBB634C4D438CF09C7BC8E97D9A] #48 in 3ms
2016-01-14 18:31:43,581 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D8EB5CBB634C4D438CF09C7BC8E97D9A] #48, status:false, isInterrupted:false
2016-01-14 18:31:43,582 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:43,582 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:43,582 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:43,582 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:43,582 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:43,582 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D8EB5CBB634C4D438CF09C7BC8E97D9A
2016-01-14 18:31:43,583 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:43,583 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BD604E693AEA4BC0B297A126BCAE3657, all 1 inputs ready in: 00:00:00.004
2016-01-14 18:31:43,583 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7d75829a; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2dd18ccf
2016-01-14 18:31:43,585 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:43,586 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:43,586 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796303313_0001_1_01_000000_0
2016-01-14 18:31:43,586 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796303313_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:43,587 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796303313_0001_1_01_000000_0
2016-01-14 18:31:43,587 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BD604E693AEA4BC0B297A126BCAE3657, mem on close (mb), free: 290, total: 563, max: 672
2016-01-14 18:31:43,587 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D8EB5CBB634C4D438CF09C7BC8E97D9A
2016-01-14 18:31:43,587 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:43,588 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:43,588 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:43,588 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:43,588 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:43 Completed running task attempt: attempt_1452796303313_0001_1_01_000000_0
2016-01-14 18:31:43,589 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796303313_0001_1_01_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:43,589 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 46 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796303583, Process_Duration=4, Process_End_Time=1452796303587, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Written=20]]
2016-01-14 18:31:43,589 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BD604E693AEA4BC0B297A126BCAE3657, taskAttemptId=attempt_1452796303313_0001_1_01_000000_0, startTime=1452796303556, finishTime=1452796303589, timeTaken=33, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796303583, Process_Duration=4, Process_End_Time=1452796303587, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:43,590 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:43,590 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:43,590 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303313_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:43,590 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,590 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:43,590 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796303313_0001_1_01_000000_0
2016-01-14 18:31:43,590 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796303313_0001_00_000002
2016-01-14 18:31:43,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:43,591 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:TASK_FINISHED]: vertexName=BD604E693AEA4BC0B297A126BCAE3657, taskId=task_1452796303313_0001_1_01_000000, startTime=1452796303556, finishTime=1452796303590, timeTaken=34, status=SUCCEEDED, successfulAttemptID=attempt_1452796303313_0001_1_01_000000_0, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796303583, Process_Duration=4, Process_End_Time=1452796303587, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20
2016-01-14 18:31:43,591 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303313_0001_1_01_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:43,591 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657] : 1
2016-01-14 18:31:43,591 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657]
2016-01-14 18:31:43,591 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:43,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:VERTEX_FINISHED]: vertexName=BD604E693AEA4BC0B297A126BCAE3657, vertexId=vertex_1452796303313_0001_1_01, initRequestedTime=1452796303476, initedTime=1452796303484, startRequestedTime=1452796303489, startedTime=1452796303489, finishTime=1452796303591, timeTaken=102, status=SUCCEEDED, diagnostics=, counters=Counters: 46, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=132, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=20, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, OUTPUT_RECORDS=20, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796303583, Process_Duration=4, Process_End_Time=1452796303587, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Written=20, vertexStats=firstTaskStartTime=1452796303556, firstTasksToStart=[ task_1452796303313_0001_1_01_000000 ], lastTaskFinishTime=1452796303589, lastTasksToFinish=[ task_1452796303313_0001_1_01_000000 ], minTaskDuration=33, maxTaskDuration=33, avgTaskDuration=33.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796303313_0001_1_01_000000 ], longestDurationTasks=[ task_1452796303313_0001_1_01_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:43,592 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:43,592 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000002 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:43,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796303313_0001_00_000002, stoppedTime=1452796303592, exitStatus=0
2016-01-14 18:31:43,592 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000002 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:43,592 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:43,592 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657] completed., numCompletedVertices=2, numSuccessfulVertices=2, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:43,592 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796303313_0001_1
2016-01-14 18:31:43,592 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:DAG_COMMIT_STARTED]: dagID=dag_1452796303313_0001_1
2016-01-14 18:31:43,592 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(892)) - Committing output: EC1960F079564C6AAC5C20B085C5BAEB for vertex: vertex_1452796303313_0001_1_01 [BD604E693AEA4BC0B297A126BCAE3657]
2016-01-14 18:31:43,597 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(871)) - No output committers for vertex: vertex_1452796303313_0001_1_00 [D8EB5CBB634C4D438CF09C7BC8E97D9A]
2016-01-14 18:31:43,599 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796303313_0001_1, queueSize=0
2016-01-14 18:31:43,599 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:43,599 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303313_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796303313_0001_1, startTime=1452796303474, finishTime=1452796303597, timeTaken=123, status=SUCCEEDED, diagnostics=, counters=Counters: 55, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=2, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=442, FILE_BYTES_WRITTEN=465, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=19, REDUCE_INPUT_RECORDS=20, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=40, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=9, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=1181745152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=40, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=301, ADDITIONAL_SPILL_COUNT=0, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=8, MERGE_PHASE_TIME=9, FIRST_EVENT_RECEIVED=5, LAST_EVENT_RECEIVED=5, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=2905592607121, Process_Duration=6, Process_End_Time=2905592607127, Read_Duration=2, Tuples_Read=40, Tuples_Written=40, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, Tuples_Written=20
2016-01-14 18:31:43,599 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796303313_0001_1 finished with state: SUCCEEDED
2016-01-14 18:31:43,599 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796303313_0001_1 transitioned from RUNNING to SUCCEEDED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:43,600 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796303313_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:43 Completed Dag: dag_1452796303313_0001_1
2016-01-14 18:31:43,600 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796303313_0001_1, dagState=SUCCEEDED
2016-01-14 18:31:43,600 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796303313_0001_1_post
2016-01-14 18:31:43,601 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,601 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:43,601 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796303313_0001_00_000002 is valid, but no longer registered, and will be killed
2016-01-14 18:31:43,601 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796303313_0001_00_000002: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:43,601 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796303313_0001_00_000002, Exiting
2016-01-14 18:31:43,601 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796303313_0001_00_000002
2016-01-14 18:31:43,601 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796303313_0001_00_000002
2016-01-14 18:31:43,601 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796303313_0001_00_000002 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:43,601 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796303313_0001_00_000002 completed successfully
2016-01-14 18:31:43,601 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:43,602 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303313_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:43,605 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_UNKNOWN_SWAP, applicationId=application_1452796303313_0001
2016-01-14 18:31:43,607 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:43,607 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:43,607 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:43,607 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:43,607 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:43,607 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:43,608 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:43,609 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:43,609 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:43,609 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:43,609 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:43,609 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:43,610 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:43,613 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/E3A8BA3CA1AC413781E5FEEE507FE670/C3E8DC311A4041818EF718D35DDD546D/.tez/application_1452796303313_0001
2016-01-14 18:31:43,614 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.317
2016-01-14 18:31:43,682 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_UNKNOWN_SWAP
2016-01-14 18:31:43,704 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:43,704 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:43,734 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-01-14 18:31:43,739 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 18:31:43,740 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:43,740 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:43,740 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:43,757 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:43,763 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1
2016-01-14 18:31:43,810 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:43,810 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:43,810 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1"]
2016-01-14 18:31:43,811 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:43,811 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:43,811 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:43,811 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 18:31:43,811 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:43,823 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:43,828 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:43,828 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:43,828 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:43,836 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6/.tez/application_1452796303828_0001 doesn't exist and is created
2016-01-14 18:31:43,849 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:43,850 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:43,850 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6/.tez/application_1452796303828_0001
2016-01-14 18:31:43,851 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796303828_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:43,862 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:43,862 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:43,863 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:43,863 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:43,863 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6/.tez/application_1452796303828_0001/localmode-log-dir
2016-01-14 18:31:43,864 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6/.tez/application_1452796303828_0001/localmode-log-dir/history.txt.appattempt_1452796303828_0001_000000, maxErrors=10
2016-01-14 18:31:43,864 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:43,864 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796303828_0001_000000, appSubmitTime=1452796303851, launchTime=1452796303851
2016-01-14 18:31:43,868 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:53391
2016-01-14 18:31:43,869 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:43,870 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:43,870 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:43,871 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:43,871 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:43,872 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796303828_0001_000000, startTime=1452796303872
2016-01-14 18:31:43,872 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:43,872 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:43,898 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:43,898 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:43,950 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:43,950 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:43,950 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1452796303828_0001, dagName=[6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 18:31:43,966 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 18:31:43,967 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796303828_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6/.tez/application_1452796303828_0001/localmode-log-dir/dag_1452796303828_0001_1.dot
2016-01-14 18:31:43,967 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6/.tez/application_1452796303828_0001/localmode-log-dir/dag_1452796303828_0001_1-tez-dag.pb.txt
2016-01-14 18:31:43,973 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796303828_0001_1
2016-01-14 18:31:43,973 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:43,973 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 18:31:43 Running Dag: dag_1452796303828_0001_1
2016-01-14 18:31:43,982 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796303828_0001_1, submitTime=1452796303966
2016-01-14 18:31:43,982 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:43,983 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex FA6E0E27C90E44C483102331C4509331
2016-01-14 18:31:43,983 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:43,983 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796303828_0001_1, initTime=1452796303983
2016-01-14 18:31:43,983 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796303828_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:43,984 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:DAG_STARTED]: dagID=dag_1452796303828_0001_1, startTime=1452796303984
2016-01-14 18:31:43,984 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:43,984 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796303828_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:43,984 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: E3F8B77D92494F5AA5DCC0AF0482C2C1 : {59E6DD7311914C9E8B4CDDD5C726E8A0={InputName=59E6DD7311914C9E8B4CDDD5C726E8A0}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:43,984 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 59E6DD7311914C9E8B4CDDD5C726E8A0, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:43,984 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1]
2016-01-14 18:31:43,984 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1]
2016-01-14 18:31:43,984 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1]
2016-01-14 18:31:43,984 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1]. Starting root input initializers: 1
2016-01-14 18:31:43,985 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:43,985 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331]
2016-01-14 18:31:43,985 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1452796303828_0001, dagName=[6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6] every/(1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 18:31:43,983 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:43,986 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:43,987 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 59E6DD7311914C9E8B4CDDD5C726E8A0 on vertex vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1]
2016-01-14 18:31:43,988 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:43,988 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331]
2016-01-14 18:31:43,988 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331]
2016-01-14 18:31:43,988 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331]
2016-01-14 18:31:43,988 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=A4E78334D5FC4345A34368CF1166BD73, vertexId=vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:43,989 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=A4E78334D5FC4345A34368CF1166BD73, vertexId=vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331]
2016-01-14 18:31:43,989 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:43,989 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 59E6DD7311914C9E8B4CDDD5C726E8A0 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:43,990 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:43,990 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:43,990 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=A4E78334D5FC4345A34368CF1166BD73, vertexName=FA6E0E27C90E44C483102331C4509331, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:43,991 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=A4E78334D5FC4345A34368CF1166BD73, vertexId=vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331]
2016-01-14 18:31:43,991 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:43,991 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:VERTEX_INITIALIZED]: vertexName=FA6E0E27C90E44C483102331C4509331, vertexId=vertex_1452796303828_0001_1_01, initRequestedTime=1452796303985, initedTime=1452796303991, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:43,991 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:43,992 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:43,992 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 59E6DD7311914C9E8B4CDDD5C726E8A0 on vertex vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1]
2016-01-14 18:31:43,992 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:43,992 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {59E6DD7311914C9E8B4CDDD5C726E8A0=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:43,993 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1] parallelism set to 1
2016-01-14 18:31:43,993 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:43,993 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796303828_0001_1_00, eventCount=1
2016-01-14 18:31:43,993 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:43,994 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E3F8B77D92494F5AA5DCC0AF0482C2C1, vertexId=vertex_1452796303828_0001_1_00, initRequestedTime=1452796303984, initedTime=1452796303993, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:43,994 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1] with distanceFromRoot: 0
2016-01-14 18:31:43,994 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:43,994 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in E3F8B77D92494F5AA5DCC0AF0482C2C1
2016-01-14 18:31:43,994 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1]
2016-01-14 18:31:43,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:43,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796303828_0001_1_00, startRequestedTime=1452796303994, startedTime=1452796303994
2016-01-14 18:31:43,995 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:43,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:43,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:TASK_STARTED]: vertexName=E3F8B77D92494F5AA5DCC0AF0482C2C1, taskId=task_1452796303828_0001_1_00_000000, scheduledTime=1452796303995, launchTime=1452796303995
2016-01-14 18:31:43,996 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303828_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:43,996 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796303828_0001_1_00 for vertex: vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] numStartedSources: 1 numSources: 1
2016-01-14 18:31:43,996 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] with distanceFromRoot: 1
2016-01-14 18:31:43,996 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1]
2016-01-14 18:31:43,996 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: E3F8B77D92494F5AA5DCC0AF0482C2C1 in vertex: FA6E0E27C90E44C483102331C4509331
2016-01-14 18:31:43,997 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: FA6E0E27C90E44C483102331C4509331 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:43,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:43,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796303828_0001_1_01, startRequestedTime=1452796303996, startedTime=1452796303996
2016-01-14 18:31:43,997 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:43,997 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: E3F8B77D92494F5AA5DCC0AF0482C2C1, VertexParallelism: 1, TaskAttemptID:attempt_1452796303828_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=59E6DD7311914C9E8B4CDDD5C726E8A0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FA6E0E27C90E44C483102331C4509331, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:43,998 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303828_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:43,998 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:43,999 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:43,999 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:43,999 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:44,000 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796303828_0001_00_000001
2016-01-14 18:31:44,000 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:44,002 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796303828_0001_00_000001
2016-01-14 18:31:44,002 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:44,003 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:44,003 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796303828_0001, with dag id: dag_1452796303828_0001_1
2016-01-14 18:31:44,003 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796303828_0001_00_000001, launchTime=1452796304003
2016-01-14 18:31:44,003 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:44,004 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796303828_0001_00_000001
2016-01-14 18:31:44,005 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796303828_0001_1_00_000000_0] to container: [container_1452796303828_0001_00_000001]
2016-01-14 18:31:44,005 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:44,005 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796303828_0001_00_000001 given task: attempt_1452796303828_0001_1_00_000000_0
2016-01-14 18:31:44,005 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796303828_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796303828_0001_1_00_000000_0
2016-01-14 18:31:44,006 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796303828_0001_1_00_000000_0] started. Is using containerId: [container_1452796303828_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:44,006 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E3F8B77D92494F5AA5DCC0AF0482C2C1, taskAttemptId=attempt_1452796303828_0001_1_00_000000_0, startTime=1452796304005, containerId=container_1452796303828_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796303828_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:44 Starting to run new task attempt: attempt_1452796303828_0001_1_00_000000_0
2016-01-14 18:31:44,006 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796303828_0001_1_00_000000_0
2016-01-14 18:31:44,006 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:44,006 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:44,006 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:44,006 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:44,006 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:44,006 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:44,006 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:44,007 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: E3F8B77D92494F5AA5DCC0AF0482C2C1, VertexParallelism: 1, TaskAttemptID:attempt_1452796303828_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=59E6DD7311914C9E8B4CDDD5C726E8A0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=FA6E0E27C90E44C483102331C4509331, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:44,007 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:44,006 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303828_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:44,007 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303828_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:44,008 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796303828_0001_1_00_000000_0
2016-01-14 18:31:44,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:44,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,009 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796303828_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:44,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=59E6DD7311914C9E8B4CDDD5C726E8A0, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:44,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:44,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 59E6DD7311914C9E8B4CDDD5C726E8A0
2016-01-14 18:31:44,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=FA6E0E27C90E44C483102331C4509331, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:44,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:44,011 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: FA6E0E27C90E44C483102331C4509331
2016-01-14 18:31:44,012 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:44,012 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: FA6E0E27C90E44C483102331C4509331
2016-01-14 18:31:44,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: FA6E0E27C90E44C483102331C4509331
2016-01-14 18:31:44,013 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 59E6DD7311914C9E8B4CDDD5C726E8A0
2016-01-14 18:31:44,013 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:44,015 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:44,015 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 59E6DD7311914C9E8B4CDDD5C726E8A0
2016-01-14 18:31:44,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 59E6DD7311914C9E8B4CDDD5C726E8A0
2016-01-14 18:31:44,016 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E3F8B77D92494F5AA5DCC0AF0482C2C1, ordinal: 0
2016-01-14 18:31:44,016 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E3F8B77D92494F5AA5DCC0AF0482C2C1, mem on start (mb), free: 182, total: 563, max: 672
2016-01-14 18:31:44,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:44,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:44,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:44,017 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:44,017 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:44,017 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:44,017 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:44,017 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:44,018 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:44,018 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, FA6E0E27C90E44C483102331C4509331, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:44,018 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 59E6DD7311914C9E8B4CDDD5C726E8A0, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:44,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:44,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 59E6DD7311914C9E8B4CDDD5C726E8A0 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:44,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:44,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 59E6DD7311914C9E8B4CDDD5C726E8A0
2016-01-14 18:31:44,018 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:44,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 59E6DD7311914C9E8B4CDDD5C726E8A0
2016-01-14 18:31:44,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:44,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:44,019 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796303828_0001_1_00_000000_0
2016-01-14 18:31:44,022 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:44,022 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 59E6DD7311914C9E8B4CDDD5C726E8A0
2016-01-14 18:31:44,022 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: AEBB657E1FE14E35B56198BF2AC8FA7A
2016-01-14 18:31:44,023 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] AEBB657E1FE14E35B56198BF2AC8FA7A
2016-01-14 18:31:44,022 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:44,023 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@6917ef5; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@3a260a31; comparator=cascading.tuple.hadoop.util.TupleComparator@176734ca; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:44,023 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:44,023 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:44,024 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:44,033 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3c0873f9
2016-01-14 18:31:44,034 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3c0873f9
2016-01-14 18:31:44,034 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:44,035 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:44,045 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:44,045 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:44,045 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:44,045 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:44,045 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:44,046 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:44,046 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:44,046 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E3F8B77D92494F5AA5DCC0AF0482C2C1, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:44,048 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796303828_0001_1_00_000000_0
2016-01-14 18:31:44,048 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E3F8B77D92494F5AA5DCC0AF0482C2C1, mem on close (mb), free: 295, total: 563, max: 672
2016-01-14 18:31:44,048 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:44,048 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:44,048 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:44,048 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:44,050 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:44 Completed running task attempt: attempt_1452796303828_0001_1_00_000000_0
2016-01-14 18:31:44,052 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796303828_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:44,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796303828_0001_1_00, eventCount=1
2016-01-14 18:31:44,053 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:44,053 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E3F8B77D92494F5AA5DCC0AF0482C2C1, taskAttemptId=attempt_1452796303828_0001_1_00_000000_0, startTime=1452796304005, finishTime=1452796304053, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796304046, Process_Duration=2, Process_End_Time=1452796304048, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:44,053 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303828_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:44,053 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:44,053 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796303828_0001_1_00_000000_0
2016-01-14 18:31:44,054 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:44,054 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:TASK_FINISHED]: vertexName=E3F8B77D92494F5AA5DCC0AF0482C2C1, taskId=task_1452796303828_0001_1_00_000000, startTime=1452796304005, finishTime=1452796304054, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452796303828_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796304046, Process_Duration=2, Process_End_Time=1452796304048, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:44,054 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303828_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:44,054 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1] : 1
2016-01-14 18:31:44,054 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1]
2016-01-14 18:31:44,055 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:44,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:VERTEX_FINISHED]: vertexName=E3F8B77D92494F5AA5DCC0AF0482C2C1, vertexId=vertex_1452796303828_0001_1_00, initRequestedTime=1452796303984, initedTime=1452796303993, startRequestedTime=1452796303994, startedTime=1452796303994, finishTime=1452796304054, timeTaken=60, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796304046, Process_Duration=2, Process_End_Time=1452796304048, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796304005, firstTasksToStart=[ task_1452796303828_0001_1_00_000000 ], lastTaskFinishTime=1452796304053, lastTasksToFinish=[ task_1452796303828_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796303828_0001_1_00_000000 ], longestDurationTasks=[ task_1452796303828_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:44,055 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:44,055 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:44,055 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796304046, Process_Duration=2, Process_End_Time=1452796304048, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:44,055 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796303828_0001_00_000001, stoppedTime=1452796304055, exitStatus=0
2016-01-14 18:31:44,055 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:44,055 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:44,055 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:44,056 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:44,056 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] attempt: attempt_1452796303828_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:44,056 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: FA6E0E27C90E44C483102331C4509331
2016-01-14 18:31:44,056 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331]
2016-01-14 18:31:44,056 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:44,056 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796303828_0001_00_000001
2016-01-14 18:31:44,057 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796303828_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:44,057 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796303828_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:44,057 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796303828_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:44,057 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796303828_0001_00_000001, Exiting
2016-01-14 18:31:44,057 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796303828_0001_00_000001
2016-01-14 18:31:44,057 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796303828_0001_00_000001
2016-01-14 18:31:44,057 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:TASK_STARTED]: vertexName=FA6E0E27C90E44C483102331C4509331, taskId=task_1452796303828_0001_1_01_000000, scheduledTime=1452796304057, launchTime=1452796304057
2016-01-14 18:31:44,057 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303828_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:44,058 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:44,058 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331]
2016-01-14 18:31:44,058 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796303828_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:44,058 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796303828_0001_00_000001 completed successfully
2016-01-14 18:31:44,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:44,058 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: FA6E0E27C90E44C483102331C4509331, VertexParallelism: 1, TaskAttemptID:attempt_1452796303828_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E3F8B77D92494F5AA5DCC0AF0482C2C1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A4E78334D5FC4345A34368CF1166BD73, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:44,059 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303828_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:44,059 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:44,059 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:44,060 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796303828_0001_00_000002
2016-01-14 18:31:44,060 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:44,062 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796303828_0001_00_000002
2016-01-14 18:31:44,063 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:44,065 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:44,065 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796303828_0001_00_000002, launchTime=1452796304065
2016-01-14 18:31:44,065 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:44,066 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796303828_0001_00_000002
2016-01-14 18:31:44,066 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796303828_0001_1_01_000000_0] to container: [container_1452796303828_0001_00_000002]
2016-01-14 18:31:44,066 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:44,067 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796303828_0001_1_01_000000_0] started. Is using containerId: [container_1452796303828_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:44,067 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=FA6E0E27C90E44C483102331C4509331, taskAttemptId=attempt_1452796303828_0001_1_01_000000_0, startTime=1452796304067, containerId=container_1452796303828_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796303828_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:44,067 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303828_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:44,067 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:44,067 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303828_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:44,067 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796303828_0001_00_000002 given task: attempt_1452796303828_0001_1_01_000000_0
2016-01-14 18:31:44,067 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796303828_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796303828_0001_1_01_000000_0
2016-01-14 18:31:44 Starting to run new task attempt: attempt_1452796303828_0001_1_01_000000_0
2016-01-14 18:31:44,068 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796303828_0001_1_01_000000_0
2016-01-14 18:31:44,068 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:44,068 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:44,068 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:44,068 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:44,068 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:44,068 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:44,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6] every/(1/1) ...blyplatform/ALL_VALUES_-1, VertexName: FA6E0E27C90E44C483102331C4509331, VertexParallelism: 1, TaskAttemptID:attempt_1452796303828_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E3F8B77D92494F5AA5DCC0AF0482C2C1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=A4E78334D5FC4345A34368CF1166BD73, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:44,069 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:44,069 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796303828_0001_1_01_000000_0
2016-01-14 18:31:44,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:44,069 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,070 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796303828_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:44,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E3F8B77D92494F5AA5DCC0AF0482C2C1, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:44,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,070 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A4E78334D5FC4345A34368CF1166BD73, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:44,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:44,071 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A4E78334D5FC4345A34368CF1166BD73
2016-01-14 18:31:44,071 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:44,071 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: A4E78334D5FC4345A34368CF1166BD73
2016-01-14 18:31:44,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:44,072 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E3F8B77D92494F5AA5DCC0AF0482C2C1
2016-01-14 18:31:44,073 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:44,075 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:44,075 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:44,075 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: E3F8B77D92494F5AA5DCC0AF0482C2C1
2016-01-14 18:31:44,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E3F8B77D92494F5AA5DCC0AF0482C2C1
2016-01-14 18:31:44,077 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: FA6E0E27C90E44C483102331C4509331, ordinal: 1
2016-01-14 18:31:44,077 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA6E0E27C90E44C483102331C4509331, mem on start (mb), free: 289, total: 563, max: 672
2016-01-14 18:31:44,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:44,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:44,080 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:44,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A4E78334D5FC4345A34368CF1166BD73
2016-01-14 18:31:44,080 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:44,080 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:44,080 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:44,080 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:44,080 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:44,080 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:44,080 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:44,081 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A4E78334D5FC4345A34368CF1166BD73, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:44,081 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E3F8B77D92494F5AA5DCC0AF0482C2C1, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:44,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:44,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E3F8B77D92494F5AA5DCC0AF0482C2C1 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:44,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:44,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:44,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E3F8B77D92494F5AA5DCC0AF0482C2C1
2016-01-14 18:31:44,081 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:44,082 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:44,082 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: E3F8B77D92494F5AA5DCC0AF0482C2C1 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:44,082 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:44,082 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:44,082 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:44,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E3F8B77D92494F5AA5DCC0AF0482C2C1
2016-01-14 18:31:44,083 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:44,083 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796303828_0001_1_01_000000_0
2016-01-14 18:31:44,084 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796303828_0001_1_00_000000_0_10291, runDuration: 40642, ]
2016-01-14 18:31:44,085 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796303828_0001_1_00_000000_0_10291] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796304085, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:44,085 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : E3F8B77D92494F5AA5DCC0AF0482C2C1
2016-01-14 18:31:44,086 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:44,086 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [E3F8B77D92494F5AA5DCC0AF0482C2C1] #49 in 2ms
2016-01-14 18:31:44,086 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [E3F8B77D92494F5AA5DCC0AF0482C2C1] #49, status:false, isInterrupted:false
2016-01-14 18:31:44,086 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:44,087 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:44,087 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:44,087 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:44,087 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: AEBB657E1FE14E35B56198BF2AC8FA7A
2016-01-14 18:31:44,087 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1"], id: A4E78334D5FC4345A34368CF1166BD73
2016-01-14 18:31:44,087 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:44,087 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_-1"]
2016-01-14 18:31:44,087 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] AEBB657E1FE14E35B56198BF2AC8FA7A, for 1 inputs
2016-01-14 18:31:44,087 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:44,088 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : E3F8B77D92494F5AA5DCC0AF0482C2C1
2016-01-14 18:31:44,088 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:44,088 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: FA6E0E27C90E44C483102331C4509331, all 1 inputs ready in: 00:00:00.001
2016-01-14 18:31:44,088 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@77d2ff; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3abd6396
2016-01-14 18:31:44,088 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:44,089 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:44,089 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:44,089 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:44,090 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796303828_0001_1_01_000000_0
2016-01-14 18:31:44,090 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796303828_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:44,090 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796303828_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:44,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796304088, Process_Duration=2, Process_End_Time=1452796304090, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:44,091 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: FA6E0E27C90E44C483102331C4509331, mem on close (mb), free: 286, total: 563, max: 672
2016-01-14 18:31:44,092 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=FA6E0E27C90E44C483102331C4509331, index=1
2016-01-14 18:31:44,091 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:44,092 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: E3F8B77D92494F5AA5DCC0AF0482C2C1
2016-01-14 18:31:44,091 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=FA6E0E27C90E44C483102331C4509331, taskAttemptId=attempt_1452796303828_0001_1_01_000000_0, startTime=1452796304067, finishTime=1452796304091, timeTaken=24, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796304088, Process_Duration=2, Process_End_Time=1452796304090, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:44,092 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303828_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:44,092 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796303828_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:44,092 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:TASK_FINISHED]: vertexName=FA6E0E27C90E44C483102331C4509331, taskId=task_1452796303828_0001_1_01_000000, startTime=1452796304067, finishTime=1452796304092, timeTaken=25, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:44,093 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:44,093 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796303828_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:44,093 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] : 1
2016-01-14 18:31:44,093 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] because task failed: task_1452796303828_0001_1_01_000000
2016-01-14 18:31:44,093 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:44,093 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:44,093 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:44,093 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331]
2016-01-14 18:31:44,093 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=A4E78334D5FC4345A34368CF1166BD73, vertexId=vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331]
2016-01-14 18:31:44,093 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:44,095 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:44,095 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:VERTEX_FINISHED]: vertexName=FA6E0E27C90E44C483102331C4509331, vertexId=vertex_1452796303828_0001_1_01, initRequestedTime=1452796303985, initedTime=1452796303991, startRequestedTime=1452796303996, startedTime=1452796303996, finishTime=1452796304094, timeTaken=98, status=FAILED, diagnostics=Task failed, taskId=task_1452796303828_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796304067, firstTasksToStart=[ task_1452796303828_0001_1_01_000000 ], lastTaskFinishTime=1452796304091, lastTasksToFinish=[ task_1452796303828_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:44,095 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:44,095 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796303828_0001_00_000002, stoppedTime=1452796304095, exitStatus=0
2016-01-14 18:31:44,095 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:44,095 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796303828_0001_1_01_000000_0 failed: true container: container_1452796303828_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:44,095 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:44,095 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:44,095 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796303828_0001_1
2016-01-14 18:31:44,096 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: A4E78334D5FC4345A34368CF1166BD73 for vertex: vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331]
2016-01-14 18:31:44,096 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796303828_0001_1_00 [E3F8B77D92494F5AA5DCC0AF0482C2C1]
2016-01-14 18:31:44,097 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796303828_0001_1, queueSize=0
2016-01-14 18:31:44,097 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796303828_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796303828_0001_1, startTime=1452796303984, finishTime=1452796304096, timeTaken=112, status=FAILED, diagnostics=Vertex failed, vertexName=FA6E0E27C90E44C483102331C4509331, vertexId=vertex_1452796303828_0001_1_01, diagnostics=[Task failed, taskId=task_1452796303828_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796303828_0001_1_01 [FA6E0E27C90E44C483102331C4509331] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=590872576, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796304046, Process_Duration=2, Process_End_Time=1452796304048, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:44,097 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796303828_0001_1 finished with state: FAILED
2016-01-14 18:31:44,097 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796303828_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:44,096 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:44,098 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:44,096 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:44,098 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:44,098 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796303828_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:44,098 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:44 Completed Dag: dag_1452796303828_0001_1
2016-01-14 18:31:44,099 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:44,100 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=FA6E0E27C90E44C483102331C4509331, sourceVertex=E3F8B77D92494F5AA5DCC0AF0482C2C1
2016-01-14 18:31:44,101 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=FA6E0E27C90E44C483102331C4509331, sourceVertex=A4E78334D5FC4345A34368CF1166BD73
2016-01-14 18:31:44,099 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796303828_0001_1, dagState=FAILED
2016-01-14 18:31:44,101 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796303828_0001_00_000002
2016-01-14 18:31:44,101 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796303828_0001_00_000002
2016-01-14 18:31:44,101 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796303828_0001_1_post
2016-01-14 18:31:44,101 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:44,101 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:44,101 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796303828_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:44,102 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796303828_0001_00_000002 completed but with errors
2016-01-14 18:31:44,102 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:44,102 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796303828_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:44,102 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796303828_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:44,102 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/ALL_VALUES_-1, applicationId=application_1452796303828_0001
2016-01-14 18:31:44,102 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796303828_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:44,104 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:44,104 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:44,104 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:44,104 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:44,104 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:44,104 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:44,105 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:44,105 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:44,105 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:44,105 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:44,105 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:44,105 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:44,105 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:44,109 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/6D983357D4734878BA4CFED9F721DDEB/E4101E9C61B14ED2803611FF90CAF4D6/.tez/application_1452796303828_0001
2016-01-14 18:31:44,109 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:44,109 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...blyplatform/ALL_VALUES_-1
2016-01-14 18:31:44,110 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:44,110 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.300
2016-01-14 18:31:44,110 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_-1
2016-01-14 18:31:44,143 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:44,144 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:44,175 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 18:31:44,177 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-01-14 18:31:44,177 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:44,177 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:44,177 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_'field'
2016-01-14 18:31:44,200 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:44,201 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:44,232 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 18:31:44,244 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 18:31:44,244 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:44,245 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:44,245 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:44,266 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:44,273 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS
2016-01-14 18:31:44,340 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:44,341 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:44,342 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS"]
2016-01-14 18:31:44,342 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:44,342 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:44,342 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:44,344 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 18:31:44,344 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:44,363 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:44,369 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:44,369 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:44,369 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:44,379 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25/.tez/application_1452796304369_0001 doesn't exist and is created
2016-01-14 18:31:44,394 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:44,394 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:44,395 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25/.tez/application_1452796304369_0001
2016-01-14 18:31:44,396 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796304369_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:44,410 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:44,411 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:44,412 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:44,412 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:44,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25/.tez/application_1452796304369_0001/localmode-log-dir
2016-01-14 18:31:44,413 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25/.tez/application_1452796304369_0001/localmode-log-dir/history.txt.appattempt_1452796304369_0001_000000, maxErrors=10
2016-01-14 18:31:44,413 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:44,414 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796304369_0001_000000, appSubmitTime=1452796304395, launchTime=1452796304396
2016-01-14 18:31:44,416 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:44,417 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:44,418 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:44,418 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:44,421 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:52622
2016-01-14 18:31:44,422 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:44,424 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796304369_0001_000000, startTime=1452796304424
2016-01-14 18:31:44,424 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:44,424 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:44,438 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:44,438 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:44,495 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:44,495 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:44,495 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1452796304369_0001, dagName=[4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 18:31:44,511 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 18:31:44,512 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796304369_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25/.tez/application_1452796304369_0001/localmode-log-dir/dag_1452796304369_0001_1.dot
2016-01-14 18:31:44,512 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25/.tez/application_1452796304369_0001/localmode-log-dir/dag_1452796304369_0001_1-tez-dag.pb.txt
2016-01-14 18:31:44,519 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796304369_0001_1
2016-01-14 18:31:44,519 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:44,519 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 18:31:44 Running Dag: dag_1452796304369_0001_1
2016-01-14 18:31:44,528 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796304369_0001_1, submitTime=1452796304511
2016-01-14 18:31:44,528 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:44,529 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex AEB4789358C4474C9088A8CBE2F9A99B
2016-01-14 18:31:44,529 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:44,529 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796304369_0001_1, initTime=1452796304528
2016-01-14 18:31:44,529 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:44,530 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796304369_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:44,530 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:DAG_STARTED]: dagID=dag_1452796304369_0001_1, startTime=1452796304530
2016-01-14 18:31:44,530 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:44,530 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:44,530 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796304369_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:44,530 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: F037E605BC14411B9A36B2A3CB8700FA : {1260932DEE6C4C0E8034002551E33748={InputName=1260932DEE6C4C0E8034002551E33748}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:44,530 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 1260932DEE6C4C0E8034002551E33748, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:44,530 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA]
2016-01-14 18:31:44,531 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA]
2016-01-14 18:31:44,531 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA]
2016-01-14 18:31:44,531 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA]. Starting root input initializers: 1
2016-01-14 18:31:44,531 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:44,531 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B]
2016-01-14 18:31:44,531 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 1260932DEE6C4C0E8034002551E33748 on vertex vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA]
2016-01-14 18:31:44,533 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:44,533 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 1260932DEE6C4C0E8034002551E33748 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:44,533 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:44,534 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:44,534 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:44,534 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B]
2016-01-14 18:31:44,534 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B]
2016-01-14 18:31:44,534 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B]
2016-01-14 18:31:44,534 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=BB94D37AC8D2455FBB6C7BBB3939DEC5, vertexId=vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:44,535 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=BB94D37AC8D2455FBB6C7BBB3939DEC5, vertexId=vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B]
2016-01-14 18:31:44,535 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:44,535 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 1260932DEE6C4C0E8034002551E33748 on vertex vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA]
2016-01-14 18:31:44,536 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1452796304369_0001, dagName=[4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25] every/(1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 18:31:44,536 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=BB94D37AC8D2455FBB6C7BBB3939DEC5, vertexName=AEB4789358C4474C9088A8CBE2F9A99B, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:44,536 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=BB94D37AC8D2455FBB6C7BBB3939DEC5, vertexId=vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B]
2016-01-14 18:31:44,537 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AEB4789358C4474C9088A8CBE2F9A99B, vertexId=vertex_1452796304369_0001_1_01, initRequestedTime=1452796304531, initedTime=1452796304537, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:44,537 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:44,538 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:44,538 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {1260932DEE6C4C0E8034002551E33748=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:44,538 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA] parallelism set to 1
2016-01-14 18:31:44,538 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:44,538 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796304369_0001_1_00, eventCount=1
2016-01-14 18:31:44,539 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:44,539 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:VERTEX_INITIALIZED]: vertexName=F037E605BC14411B9A36B2A3CB8700FA, vertexId=vertex_1452796304369_0001_1_00, initRequestedTime=1452796304530, initedTime=1452796304539, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:44,539 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA] with distanceFromRoot: 0
2016-01-14 18:31:44,539 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:44,539 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:44,540 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in F037E605BC14411B9A36B2A3CB8700FA
2016-01-14 18:31:44,540 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA]
2016-01-14 18:31:44,540 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796304369_0001_1_00, startRequestedTime=1452796304540, startedTime=1452796304540
2016-01-14 18:31:44,540 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:44,540 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:44,541 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:44,541 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:TASK_STARTED]: vertexName=F037E605BC14411B9A36B2A3CB8700FA, taskId=task_1452796304369_0001_1_00_000000, scheduledTime=1452796304541, launchTime=1452796304541
2016-01-14 18:31:44,541 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304369_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:44,541 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796304369_0001_1_00 for vertex: vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] numStartedSources: 1 numSources: 1
2016-01-14 18:31:44,541 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] with distanceFromRoot: 1
2016-01-14 18:31:44,541 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA]
2016-01-14 18:31:44,542 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: F037E605BC14411B9A36B2A3CB8700FA in vertex: AEB4789358C4474C9088A8CBE2F9A99B
2016-01-14 18:31:44,542 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: AEB4789358C4474C9088A8CBE2F9A99B with 1 source tasks and 1 pending tasks
2016-01-14 18:31:44,542 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796304369_0001_1_01, startRequestedTime=1452796304542, startedTime=1452796304542
2016-01-14 18:31:44,542 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:44,542 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:44,543 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: F037E605BC14411B9A36B2A3CB8700FA, VertexParallelism: 1, TaskAttemptID:attempt_1452796304369_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1260932DEE6C4C0E8034002551E33748, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AEB4789358C4474C9088A8CBE2F9A99B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:44,543 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304369_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:44,543 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:44,544 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:44,544 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:44,544 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:44,545 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796304369_0001_00_000001
2016-01-14 18:31:44,545 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:44,546 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796304369_0001_00_000001
2016-01-14 18:31:44,546 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:44,547 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:44,547 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:44,547 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796304369_0001_00_000001, launchTime=1452796304547
2016-01-14 18:31:44,548 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796304369_0001_00_000001
2016-01-14 18:31:44,548 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796304369_0001_1_00_000000_0] to container: [container_1452796304369_0001_00_000001]
2016-01-14 18:31:44,548 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:44,549 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796304369_0001_00_000001 given task: attempt_1452796304369_0001_1_00_000000_0
2016-01-14 18:31:44,549 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796304369_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796304369_0001_1_00_000000_0
2016-01-14 18:31:44,549 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796304369_0001_1_00_000000_0] started. Is using containerId: [container_1452796304369_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:44 Starting to run new task attempt: attempt_1452796304369_0001_1_00_000000_0
2016-01-14 18:31:44,549 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796304369_0001_1_00_000000_0
2016-01-14 18:31:44,549 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:44,550 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:44,550 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:44,550 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:44,550 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:44,550 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:44,550 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: F037E605BC14411B9A36B2A3CB8700FA, VertexParallelism: 1, TaskAttemptID:attempt_1452796304369_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1260932DEE6C4C0E8034002551E33748, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=AEB4789358C4474C9088A8CBE2F9A99B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:44,550 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:44,550 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:44,550 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=F037E605BC14411B9A36B2A3CB8700FA, taskAttemptId=attempt_1452796304369_0001_1_00_000000_0, startTime=1452796304549, containerId=container_1452796304369_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796304369_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:44,551 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304369_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:44,551 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304369_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:44,551 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796304369_0001_1_00_000000_0
2016-01-14 18:31:44,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:44,552 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,553 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796304369_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:44,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=1260932DEE6C4C0E8034002551E33748, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:44,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,553 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796304369_0001, with dag id: dag_1452796304369_0001_1
2016-01-14 18:31:44,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:44,555 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 1260932DEE6C4C0E8034002551E33748
2016-01-14 18:31:44,555 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 1260932DEE6C4C0E8034002551E33748
2016-01-14 18:31:44,555 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:44,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=AEB4789358C4474C9088A8CBE2F9A99B, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:44,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:44,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: AEB4789358C4474C9088A8CBE2F9A99B
2016-01-14 18:31:44,557 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:44,557 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 1260932DEE6C4C0E8034002551E33748
2016-01-14 18:31:44,557 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 1260932DEE6C4C0E8034002551E33748
2016-01-14 18:31:44,560 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:44,560 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: AEB4789358C4474C9088A8CBE2F9A99B
2016-01-14 18:31:44,560 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: AEB4789358C4474C9088A8CBE2F9A99B
2016-01-14 18:31:44,560 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: F037E605BC14411B9A36B2A3CB8700FA, ordinal: 0
2016-01-14 18:31:44,561 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F037E605BC14411B9A36B2A3CB8700FA, mem on start (mb), free: 180, total: 563, max: 672
2016-01-14 18:31:44,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,561 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:44,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:44,562 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:44,562 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:44,563 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:44,563 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:44,563 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:44,563 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:44,563 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:44,564 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 1260932DEE6C4C0E8034002551E33748, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:44,564 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, AEB4789358C4474C9088A8CBE2F9A99B, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:44,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:44,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 1260932DEE6C4C0E8034002551E33748 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:44,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 1260932DEE6C4C0E8034002551E33748
2016-01-14 18:31:44,564 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:44,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 1260932DEE6C4C0E8034002551E33748
2016-01-14 18:31:44,564 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:44,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:44,565 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:44,565 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796304369_0001_1_00_000000_0
2016-01-14 18:31:44,566 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:44,566 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@4655525f
2016-01-14 18:31:44,567 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@4655525f
2016-01-14 18:31:44,567 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:44,568 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:44,568 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:44,568 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 1260932DEE6C4C0E8034002551E33748
2016-01-14 18:31:44,568 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 1926569B79274D9A88F74F71BF6D4C45
2016-01-14 18:31:44,569 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 1926569B79274D9A88F74F71BF6D4C45
2016-01-14 18:31:44,569 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@4d83fd7; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@11473ebd; comparator=cascading.tuple.hadoop.util.TupleComparator@5c67f327; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:44,569 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:44,569 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:44,569 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:44,595 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:44,596 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:44,596 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:44,596 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:44,596 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:44,596 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:44,597 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:44,597 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: F037E605BC14411B9A36B2A3CB8700FA, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:44,600 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796304369_0001_1_00_000000_0
2016-01-14 18:31:44,600 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: F037E605BC14411B9A36B2A3CB8700FA, mem on close (mb), free: 293, total: 564, max: 672
2016-01-14 18:31:44,601 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:44,601 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:44,601 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:44,601 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:44,603 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:44 Completed running task attempt: attempt_1452796304369_0001_1_00_000000_0
2016-01-14 18:31:44,605 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796304369_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:44,605 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796304597, Process_Duration=3, Process_End_Time=1452796304600, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:44,606 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796304369_0001_1_00, eventCount=1
2016-01-14 18:31:44,606 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:44,606 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:44,606 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796304369_0001_00_000001
2016-01-14 18:31:44,607 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:44,607 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=F037E605BC14411B9A36B2A3CB8700FA, taskAttemptId=attempt_1452796304369_0001_1_00_000000_0, startTime=1452796304549, finishTime=1452796304606, timeTaken=57, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796304597, Process_Duration=3, Process_End_Time=1452796304600, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:44,608 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304369_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:44,608 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:44,608 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796304369_0001_1_00_000000_0
2016-01-14 18:31:44,608 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:44,609 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:TASK_FINISHED]: vertexName=F037E605BC14411B9A36B2A3CB8700FA, taskId=task_1452796304369_0001_1_00_000000, startTime=1452796304549, finishTime=1452796304608, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1452796304369_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796304597, Process_Duration=3, Process_End_Time=1452796304600, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:44,609 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304369_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:44,609 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:44,610 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:44,610 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796304369_0001_00_000001, stoppedTime=1452796304610, exitStatus=0
2016-01-14 18:31:44,610 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:44,610 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA] : 1
2016-01-14 18:31:44,610 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA]
2016-01-14 18:31:44,611 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:44,612 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:VERTEX_FINISHED]: vertexName=F037E605BC14411B9A36B2A3CB8700FA, vertexId=vertex_1452796304369_0001_1_00, initRequestedTime=1452796304530, initedTime=1452796304539, startRequestedTime=1452796304540, startedTime=1452796304540, finishTime=1452796304610, timeTaken=70, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796304597, Process_Duration=3, Process_End_Time=1452796304600, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796304549, firstTasksToStart=[ task_1452796304369_0001_1_00_000000 ], lastTaskFinishTime=1452796304606, lastTasksToFinish=[ task_1452796304369_0001_1_00_000000 ], minTaskDuration=57, maxTaskDuration=57, avgTaskDuration=57.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796304369_0001_1_00_000000 ], longestDurationTasks=[ task_1452796304369_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:44,612 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:44,612 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796304369_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:44,612 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] attempt: attempt_1452796304369_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:44,612 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: AEB4789358C4474C9088A8CBE2F9A99B
2016-01-14 18:31:44,612 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B]
2016-01-14 18:31:44,613 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:44,613 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:44,613 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:44,613 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:TASK_STARTED]: vertexName=AEB4789358C4474C9088A8CBE2F9A99B, taskId=task_1452796304369_0001_1_01_000000, scheduledTime=1452796304613, launchTime=1452796304613
2016-01-14 18:31:44,614 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304369_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:44,614 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B]
2016-01-14 18:31:44,614 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: AEB4789358C4474C9088A8CBE2F9A99B, VertexParallelism: 1, TaskAttemptID:attempt_1452796304369_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F037E605BC14411B9A36B2A3CB8700FA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BB94D37AC8D2455FBB6C7BBB3939DEC5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:44,614 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304369_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:44,614 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:44,615 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796304369_0001_00_000002
2016-01-14 18:31:44,615 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:44,616 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796304369_0001_00_000002
2016-01-14 18:31:44,617 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:44,617 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796304369_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:44,617 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796304369_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:44,617 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796304369_0001_00_000001, Exiting
2016-01-14 18:31:44,617 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796304369_0001_00_000001
2016-01-14 18:31:44,617 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796304369_0001_00_000001
2016-01-14 18:31:44,617 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:44,618 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796304369_0001_00_000002, launchTime=1452796304617
2016-01-14 18:31:44,618 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:44,618 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796304369_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:44,619 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796304369_0001_00_000002
2016-01-14 18:31:44,619 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796304369_0001_00_000001 completed successfully
2016-01-14 18:31:44,619 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796304369_0001_1_01_000000_0] to container: [container_1452796304369_0001_00_000002]
2016-01-14 18:31:44,619 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:44,619 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796304369_0001_00_000002 given task: attempt_1452796304369_0001_1_01_000000_0
2016-01-14 18:31:44,619 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:44,619 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796304369_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796304369_0001_1_01_000000_0
2016-01-14 18:31:44,620 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796304369_0001_1_01_000000_0] started. Is using containerId: [container_1452796304369_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:44,620 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AEB4789358C4474C9088A8CBE2F9A99B, taskAttemptId=attempt_1452796304369_0001_1_01_000000_0, startTime=1452796304620, containerId=container_1452796304369_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796304369_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:44,620 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:44,621 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304369_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:44,621 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304369_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:44 Starting to run new task attempt: attempt_1452796304369_0001_1_01_000000_0
2016-01-14 18:31:44,621 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796304369_0001_1_01_000000_0
2016-01-14 18:31:44,621 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:44,622 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:44,622 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:44,622 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:44,622 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:44,622 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:44,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25] every/(1/1) ...atform/ALL_VALUES_RESULTS, VertexName: AEB4789358C4474C9088A8CBE2F9A99B, VertexParallelism: 1, TaskAttemptID:attempt_1452796304369_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F037E605BC14411B9A36B2A3CB8700FA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=BB94D37AC8D2455FBB6C7BBB3939DEC5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:44,623 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:44,623 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796304369_0001_1_01_000000_0
2016-01-14 18:31:44,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:44,623 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,624 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F037E605BC14411B9A36B2A3CB8700FA, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:44,624 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BB94D37AC8D2455FBB6C7BBB3939DEC5, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:44,624 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796304369_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:44,624 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:44,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F037E605BC14411B9A36B2A3CB8700FA
2016-01-14 18:31:44,624 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,625 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:44,626 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BB94D37AC8D2455FBB6C7BBB3939DEC5
2016-01-14 18:31:44,626 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:44,626 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: BB94D37AC8D2455FBB6C7BBB3939DEC5
2016-01-14 18:31:44,627 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:44,627 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:44,627 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: F037E605BC14411B9A36B2A3CB8700FA
2016-01-14 18:31:44,627 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F037E605BC14411B9A36B2A3CB8700FA
2016-01-14 18:31:44,628 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:44,632 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AEB4789358C4474C9088A8CBE2F9A99B, ordinal: 1
2016-01-14 18:31:44,632 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AEB4789358C4474C9088A8CBE2F9A99B, mem on start (mb), free: 287, total: 564, max: 672
2016-01-14 18:31:44,632 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:44,632 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:44,633 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:44,634 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:44,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BB94D37AC8D2455FBB6C7BBB3939DEC5
2016-01-14 18:31:44,634 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:44,634 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:44,634 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:44,634 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:44,635 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:44,635 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:44,635 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:44,635 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BB94D37AC8D2455FBB6C7BBB3939DEC5, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:44,635 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F037E605BC14411B9A36B2A3CB8700FA, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:44,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:44,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F037E605BC14411B9A36B2A3CB8700FA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:44,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F037E605BC14411B9A36B2A3CB8700FA
2016-01-14 18:31:44,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:44,636 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:44,636 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:44,637 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:44,637 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: F037E605BC14411B9A36B2A3CB8700FA with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:44,637 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:44,637 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:44,637 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:44,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F037E605BC14411B9A36B2A3CB8700FA
2016-01-14 18:31:44,638 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:44,638 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796304369_0001_1_01_000000_0
2016-01-14 18:31:44,639 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796304369_0001_1_00_000000_0_10297, runDuration: 46869, ]
2016-01-14 18:31:44,640 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796304369_0001_1_00_000000_0_10297] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796304640, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:44,640 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : F037E605BC14411B9A36B2A3CB8700FA
2016-01-14 18:31:44,641 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:44,641 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [F037E605BC14411B9A36B2A3CB8700FA] #50 in 1ms
2016-01-14 18:31:44,641 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:44,641 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [F037E605BC14411B9A36B2A3CB8700FA] #50, status:false, isInterrupted:false
2016-01-14 18:31:44,642 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 1926569B79274D9A88F74F71BF6D4C45
2016-01-14 18:31:44,642 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS"], id: BB94D37AC8D2455FBB6C7BBB3939DEC5
2016-01-14 18:31:44,642 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_RESULTS"]
2016-01-14 18:31:44,642 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 1926569B79274D9A88F74F71BF6D4C45, for 1 inputs
2016-01-14 18:31:44,642 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:44,643 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:44,643 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:44,643 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:44,643 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:44,643 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : F037E605BC14411B9A36B2A3CB8700FA
2016-01-14 18:31:44,644 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:44,644 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AEB4789358C4474C9088A8CBE2F9A99B, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:44,644 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@7171f9da; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@71ca9165
2016-01-14 18:31:44,645 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:44,649 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:44,652 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:44,652 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:44,653 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796304369_0001_1_01_000000_0
2016-01-14 18:31:44,653 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796304369_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:44,653 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796304369_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:44,657 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796304644, Process_Duration=9, Process_End_Time=1452796304653, Read_Duration=1, Tuples_Read=1]]
2016-01-14 18:31:44,657 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:44,657 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AEB4789358C4474C9088A8CBE2F9A99B, taskAttemptId=attempt_1452796304369_0001_1_01_000000_0, startTime=1452796304620, finishTime=1452796304657, timeTaken=37, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796304644, Process_Duration=9, Process_End_Time=1452796304653, Read_Duration=1, Tuples_Read=1
2016-01-14 18:31:44,657 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304369_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:44,657 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796304369_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:44,657 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AEB4789358C4474C9088A8CBE2F9A99B, mem on close (mb), free: 284, total: 564, max: 672
2016-01-14 18:31:44,658 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:44,658 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=AEB4789358C4474C9088A8CBE2F9A99B, index=1
2016-01-14 18:31:44,658 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:TASK_FINISHED]: vertexName=AEB4789358C4474C9088A8CBE2F9A99B, taskId=task_1452796304369_0001_1_01_000000, startTime=1452796304620, finishTime=1452796304658, timeTaken=38, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:44,658 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304369_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:44,658 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] : 1
2016-01-14 18:31:44,658 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] because task failed: task_1452796304369_0001_1_01_000000
2016-01-14 18:31:44,658 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:44,658 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:44,658 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B]
2016-01-14 18:31:44,658 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=BB94D37AC8D2455FBB6C7BBB3939DEC5, vertexId=vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B]
2016-01-14 18:31:44,659 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:44,660 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: F037E605BC14411B9A36B2A3CB8700FA
2016-01-14 18:31:44,660 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:VERTEX_FINISHED]: vertexName=AEB4789358C4474C9088A8CBE2F9A99B, vertexId=vertex_1452796304369_0001_1_01, initRequestedTime=1452796304531, initedTime=1452796304537, startRequestedTime=1452796304542, startedTime=1452796304542, finishTime=1452796304659, timeTaken=117, status=FAILED, diagnostics=Task failed, taskId=task_1452796304369_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796304620, firstTasksToStart=[ task_1452796304369_0001_1_01_000000 ], lastTaskFinishTime=1452796304657, lastTasksToFinish=[ task_1452796304369_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:44,660 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:44,660 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796304369_0001_00_000002, stoppedTime=1452796304660, exitStatus=0
2016-01-14 18:31:44,660 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:44,660 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796304369_0001_1_01_000000_0 failed: true container: container_1452796304369_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:44,660 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:44,661 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:44,661 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796304369_0001_1
2016-01-14 18:31:44,661 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796304369_0001_1_00 [F037E605BC14411B9A36B2A3CB8700FA]
2016-01-14 18:31:44,661 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: BB94D37AC8D2455FBB6C7BBB3939DEC5 for vertex: vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B]
2016-01-14 18:31:44,660 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:44,660 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:44,661 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:44,662 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:44,662 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:44,662 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:44,662 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796304369_0001_1, queueSize=0
2016-01-14 18:31:44,662 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:44,662 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304369_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796304369_0001_1, startTime=1452796304530, finishTime=1452796304661, timeTaken=131, status=FAILED, diagnostics=Vertex failed, vertexName=AEB4789358C4474C9088A8CBE2F9A99B, vertexId=vertex_1452796304369_0001_1_01, diagnostics=[Task failed, taskId=task_1452796304369_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796304369_0001_1_01 [AEB4789358C4474C9088A8CBE2F9A99B] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796304597, Process_Duration=3, Process_End_Time=1452796304600, Read_Duration=2, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:44,662 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796304369_0001_1 finished with state: FAILED
2016-01-14 18:31:44,662 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796304369_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:44,663 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796304369_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:44 Completed Dag: dag_1452796304369_0001_1
2016-01-14 18:31:44,663 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796304369_0001_1, dagState=FAILED
2016-01-14 18:31:44,663 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796304369_0001_1_post
2016-01-14 18:31:44,663 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:44,663 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:44,664 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:44,662 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:44,664 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=AEB4789358C4474C9088A8CBE2F9A99B, sourceVertex=F037E605BC14411B9A36B2A3CB8700FA
2016-01-14 18:31:44,664 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=AEB4789358C4474C9088A8CBE2F9A99B, sourceVertex=BB94D37AC8D2455FBB6C7BBB3939DEC5
2016-01-14 18:31:44,665 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796304369_0001_00_000002
2016-01-14 18:31:44,665 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796304369_0001_00_000002
2016-01-14 18:31:44,665 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796304369_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:44,665 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796304369_0001_00_000002 completed but with errors
2016-01-14 18:31:44,665 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796304369_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:44,665 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304369_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:44,666 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304369_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:44,667 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/ALL_VALUES_RESULTS, applicationId=application_1452796304369_0001
2016-01-14 18:31:44,669 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:44,669 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:44,669 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:44,669 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:44,669 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:44,670 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:44,670 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:44,670 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:44,670 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:44,670 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:44,670 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:44,671 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:44,670 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:44,676 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/4207FD4F9CC04E25A3C85EACA108F82C/F2A02902FC764690BC51064EBFE62F25/.tez/application_1452796304369_0001
2016-01-14 18:31:44,676 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:44,676 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...atform/ALL_VALUES_RESULTS
2016-01-14 18:31:44,677 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:44,677 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.337
2016-01-14 18:31:44,677 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_RESULTS
2016-01-14 18:31:44,699 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:44,699 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:44,728 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 18:31:44,734 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.012
2016-01-14 18:31:44,735 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:44,735 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:44,735 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:44,751 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:44,756 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL
2016-01-14 18:31:44,802 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:44,802 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:44,802 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL"]
2016-01-14 18:31:44,802 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:44,802 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:44,802 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:44,803 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 18:31:44,803 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:44,815 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:44,820 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:44,820 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:44,820 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:44,828 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53/.tez/application_1452796304820_0001 doesn't exist and is created
2016-01-14 18:31:44,841 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:44,841 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:44,841 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53/.tez/application_1452796304820_0001
2016-01-14 18:31:44,843 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796304820_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:44,852 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:44,853 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:44,854 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:44,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:44,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53/.tez/application_1452796304820_0001/localmode-log-dir
2016-01-14 18:31:44,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53/.tez/application_1452796304820_0001/localmode-log-dir/history.txt.appattempt_1452796304820_0001_000000, maxErrors=10
2016-01-14 18:31:44,854 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:44,855 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796304820_0001_000000, appSubmitTime=1452796304842, launchTime=1452796304842
2016-01-14 18:31:44,859 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:35775
2016-01-14 18:31:44,861 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:44,861 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:44,862 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:44,862 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:44,864 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:44,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796304820_0001_000000, startTime=1452796304865
2016-01-14 18:31:44,865 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:44,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:44,942 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:44,942 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:44,942 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:44,942 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:44,942 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1452796304820_0001, dagName=[80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 18:31:44,959 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 18:31:44,961 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796304820_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53/.tez/application_1452796304820_0001/localmode-log-dir/dag_1452796304820_0001_1.dot
2016-01-14 18:31:44,961 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53/.tez/application_1452796304820_0001/localmode-log-dir/dag_1452796304820_0001_1-tez-dag.pb.txt
2016-01-14 18:31:44,969 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796304820_0001_1
2016-01-14 18:31:44,969 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:44,969 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 18:31:44 Running Dag: dag_1452796304820_0001_1
2016-01-14 18:31:44,978 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:44,978 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796304820_0001_1, submitTime=1452796304959
2016-01-14 18:31:44,979 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex A244E91154794479B2F018E69B105523
2016-01-14 18:31:44,979 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:44,979 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:44,979 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796304820_0001_1, initTime=1452796304978
2016-01-14 18:31:44,980 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796304820_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:44,980 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:DAG_STARTED]: dagID=dag_1452796304820_0001_1, startTime=1452796304980
2016-01-14 18:31:44,980 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:44,980 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796304820_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:44,980 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: A3067317D9EC4856AD4C276687EEC110 : {8249D382D2B2402F8CD62D10809D88AA={InputName=8249D382D2B2402F8CD62D10809D88AA}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:44,980 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 8249D382D2B2402F8CD62D10809D88AA, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:44,980 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110]
2016-01-14 18:31:44,980 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110]
2016-01-14 18:31:44,980 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110]
2016-01-14 18:31:44,980 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:44,981 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110]. Starting root input initializers: 1
2016-01-14 18:31:44,981 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:44,981 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 8249D382D2B2402F8CD62D10809D88AA on vertex vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110]
2016-01-14 18:31:44,981 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523]
2016-01-14 18:31:44,983 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:44,983 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 8249D382D2B2402F8CD62D10809D88AA asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:44,983 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:44,983 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:44,984 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1452796304820_0001, dagName=[80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53] every/(1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 18:31:44,984 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:44,984 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523]
2016-01-14 18:31:44,984 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523]
2016-01-14 18:31:44,984 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523]
2016-01-14 18:31:44,984 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=7EBDCFE34E83486794DA8904FE8352B9, vertexId=vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:44,985 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=7EBDCFE34E83486794DA8904FE8352B9, vertexId=vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523]
2016-01-14 18:31:44,985 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:44,985 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 8249D382D2B2402F8CD62D10809D88AA on vertex vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110]
2016-01-14 18:31:44,986 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=7EBDCFE34E83486794DA8904FE8352B9, vertexName=A244E91154794479B2F018E69B105523, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:44,986 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=7EBDCFE34E83486794DA8904FE8352B9, vertexId=vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523]
2016-01-14 18:31:44,987 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:44,987 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A244E91154794479B2F018E69B105523, vertexId=vertex_1452796304820_0001_1_01, initRequestedTime=1452796304981, initedTime=1452796304987, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:44,987 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:44,987 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:44,987 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {8249D382D2B2402F8CD62D10809D88AA=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:44,988 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110] parallelism set to 1
2016-01-14 18:31:44,988 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:44,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796304820_0001_1_00, eventCount=1
2016-01-14 18:31:44,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:VERTEX_INITIALIZED]: vertexName=A3067317D9EC4856AD4C276687EEC110, vertexId=vertex_1452796304820_0001_1_00, initRequestedTime=1452796304980, initedTime=1452796304988, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:44,989 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110] with distanceFromRoot: 0
2016-01-14 18:31:44,988 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:44,989 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:44,989 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in A3067317D9EC4856AD4C276687EEC110
2016-01-14 18:31:44,989 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110]
2016-01-14 18:31:44,989 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:44,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796304820_0001_1_00, startRequestedTime=1452796304989, startedTime=1452796304989
2016-01-14 18:31:44,990 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:44,990 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:44,990 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:TASK_STARTED]: vertexName=A3067317D9EC4856AD4C276687EEC110, taskId=task_1452796304820_0001_1_00_000000, scheduledTime=1452796304990, launchTime=1452796304990
2016-01-14 18:31:44,990 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304820_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:44,991 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796304820_0001_1_00 for vertex: vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] numStartedSources: 1 numSources: 1
2016-01-14 18:31:44,991 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] with distanceFromRoot: 1
2016-01-14 18:31:44,991 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110]
2016-01-14 18:31:44,991 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: A3067317D9EC4856AD4C276687EEC110 in vertex: A244E91154794479B2F018E69B105523
2016-01-14 18:31:44,991 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: A244E91154794479B2F018E69B105523 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:44,992 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:44,992 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796304820_0001_1_01, startRequestedTime=1452796304991, startedTime=1452796304991
2016-01-14 18:31:44,992 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:44,992 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: A3067317D9EC4856AD4C276687EEC110, VertexParallelism: 1, TaskAttemptID:attempt_1452796304820_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8249D382D2B2402F8CD62D10809D88AA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A244E91154794479B2F018E69B105523, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:44,992 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304820_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:44,992 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:44,993 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:44,993 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:44,994 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:44,994 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796304820_0001_00_000001
2016-01-14 18:31:44,994 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:44,996 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796304820_0001_00_000001
2016-01-14 18:31:44,996 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:44,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:44,997 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:44,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796304820_0001_00_000001, launchTime=1452796304997
2016-01-14 18:31:44,998 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796304820_0001_00_000001
2016-01-14 18:31:44,998 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796304820_0001_1_00_000000_0] to container: [container_1452796304820_0001_00_000001]
2016-01-14 18:31:44,998 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:44,999 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796304820_0001_1_00_000000_0] started. Is using containerId: [container_1452796304820_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:44,999 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A3067317D9EC4856AD4C276687EEC110, taskAttemptId=attempt_1452796304820_0001_1_00_000000_0, startTime=1452796304998, containerId=container_1452796304820_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796304820_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:44,999 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304820_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:44,999 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:44,999 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304820_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:44,999 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796304820_0001_00_000001 given task: attempt_1452796304820_0001_1_00_000000_0
2016-01-14 18:31:44,999 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796304820_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796304820_0001_1_00_000000_0
2016-01-14 18:31:45 Starting to run new task attempt: attempt_1452796304820_0001_1_00_000000_0
2016-01-14 18:31:45,000 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796304820_0001_1_00_000000_0
2016-01-14 18:31:45,000 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,000 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:45,000 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:45,001 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:45,001 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:45,001 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:45,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: A3067317D9EC4856AD4C276687EEC110, VertexParallelism: 1, TaskAttemptID:attempt_1452796304820_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=8249D382D2B2402F8CD62D10809D88AA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=A244E91154794479B2F018E69B105523, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:45,001 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:45,001 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796304820_0001, with dag id: dag_1452796304820_0001_1
2016-01-14 18:31:45,002 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796304820_0001_1_00_000000_0
2016-01-14 18:31:45,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:45,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,003 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796304820_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:45,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=A244E91154794479B2F018E69B105523, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:45,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:45,005 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: A244E91154794479B2F018E69B105523
2016-01-14 18:31:45,006 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:45,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=8249D382D2B2402F8CD62D10809D88AA, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:45,008 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:45,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 8249D382D2B2402F8CD62D10809D88AA
2016-01-14 18:31:45,009 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 8249D382D2B2402F8CD62D10809D88AA
2016-01-14 18:31:45,010 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A3067317D9EC4856AD4C276687EEC110, ordinal: 0
2016-01-14 18:31:45,010 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3067317D9EC4856AD4C276687EEC110, mem on start (mb), free: 196, total: 564, max: 672
2016-01-14 18:31:45,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:45,011 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:45,011 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: A244E91154794479B2F018E69B105523
2016-01-14 18:31:45,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: A244E91154794479B2F018E69B105523
2016-01-14 18:31:45,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:45,012 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:45,012 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 8249D382D2B2402F8CD62D10809D88AA
2016-01-14 18:31:45,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 8249D382D2B2402F8CD62D10809D88AA
2016-01-14 18:31:45,013 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:45,013 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:45,013 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:45,013 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:45,014 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:45,014 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:45,014 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:45,015 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 8249D382D2B2402F8CD62D10809D88AA, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:45,015 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, A244E91154794479B2F018E69B105523, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:45,015 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:45,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 8249D382D2B2402F8CD62D10809D88AA
2016-01-14 18:31:45,016 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:45,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 8249D382D2B2402F8CD62D10809D88AA
2016-01-14 18:31:45,016 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 8249D382D2B2402F8CD62D10809D88AA being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:45,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:45,017 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:45,018 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:45,018 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796304820_0001_1_00_000000_0
2016-01-14 18:31:45,018 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:45,019 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@762b57d0
2016-01-14 18:31:45,020 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@762b57d0
2016-01-14 18:31:45,020 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:45,020 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:45,022 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:45,023 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 8249D382D2B2402F8CD62D10809D88AA
2016-01-14 18:31:45,023 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: FF85E0AB86F64AACB485CF913752208F
2016-01-14 18:31:45,024 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] FF85E0AB86F64AACB485CF913752208F
2016-01-14 18:31:45,024 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2600225; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6d2ccc5a; comparator=cascading.tuple.hadoop.util.TupleComparator@181c60b5; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:45,025 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:45,025 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:45,025 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:45,049 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:45,049 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:45,049 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:45,050 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:45,050 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:45,051 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:45,051 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:45,051 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A3067317D9EC4856AD4C276687EEC110, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:45,054 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796304820_0001_1_00_000000_0
2016-01-14 18:31:45,054 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A3067317D9EC4856AD4C276687EEC110, mem on close (mb), free: 291, total: 564, max: 672
2016-01-14 18:31:45,055 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:45,055 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:45,055 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:45,055 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:45,056 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:45 Completed running task attempt: attempt_1452796304820_0001_1_00_000000_0
2016-01-14 18:31:45,059 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796304820_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:45,059 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796305051, Process_Duration=3, Process_End_Time=1452796305054, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:45,059 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796304820_0001_1_00, eventCount=1
2016-01-14 18:31:45,060 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:45,060 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,060 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796304820_0001_00_000001
2016-01-14 18:31:45,060 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A3067317D9EC4856AD4C276687EEC110, taskAttemptId=attempt_1452796304820_0001_1_00_000000_0, startTime=1452796304998, finishTime=1452796305060, timeTaken=62, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305051, Process_Duration=3, Process_End_Time=1452796305054, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:45,060 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304820_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:45,060 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:45,060 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796304820_0001_1_00_000000_0
2016-01-14 18:31:45,061 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:45,061 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:TASK_FINISHED]: vertexName=A3067317D9EC4856AD4C276687EEC110, taskId=task_1452796304820_0001_1_00_000000, startTime=1452796304998, finishTime=1452796305060, timeTaken=62, status=SUCCEEDED, successfulAttemptID=attempt_1452796304820_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305051, Process_Duration=3, Process_End_Time=1452796305054, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:45,061 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304820_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:45,061 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:45,061 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110] : 1
2016-01-14 18:31:45,061 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110]
2016-01-14 18:31:45,062 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:45,062 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:VERTEX_FINISHED]: vertexName=A3067317D9EC4856AD4C276687EEC110, vertexId=vertex_1452796304820_0001_1_00, initRequestedTime=1452796304980, initedTime=1452796304988, startRequestedTime=1452796304989, startedTime=1452796304989, finishTime=1452796305061, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305051, Process_Duration=3, Process_End_Time=1452796305054, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796304998, firstTasksToStart=[ task_1452796304820_0001_1_00_000000 ], lastTaskFinishTime=1452796305060, lastTasksToFinish=[ task_1452796304820_0001_1_00_000000 ], minTaskDuration=62, maxTaskDuration=62, avgTaskDuration=62.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796304820_0001_1_00_000000 ], longestDurationTasks=[ task_1452796304820_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:45,062 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:45,062 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] attempt: attempt_1452796304820_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:45,062 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: A244E91154794479B2F018E69B105523
2016-01-14 18:31:45,062 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523]
2016-01-14 18:31:45,063 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:45,063 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796304820_0001_00_000001, stoppedTime=1452796305063, exitStatus=0
2016-01-14 18:31:45,063 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:45,063 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:45,063 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:45,064 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:45,064 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:TASK_STARTED]: vertexName=A244E91154794479B2F018E69B105523, taskId=task_1452796304820_0001_1_01_000000, scheduledTime=1452796305063, launchTime=1452796305063
2016-01-14 18:31:45,064 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304820_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:45,064 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796304820_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:45,064 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523]
2016-01-14 18:31:45,064 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:45,065 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: A244E91154794479B2F018E69B105523, VertexParallelism: 1, TaskAttemptID:attempt_1452796304820_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A3067317D9EC4856AD4C276687EEC110, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7EBDCFE34E83486794DA8904FE8352B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:45,065 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304820_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:45,065 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796304820_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:45,065 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:45,065 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796304820_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:45,065 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796304820_0001_00_000001, Exiting
2016-01-14 18:31:45,065 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796304820_0001_00_000001
2016-01-14 18:31:45,065 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796304820_0001_00_000001
2016-01-14 18:31:45,066 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796304820_0001_00_000002
2016-01-14 18:31:45,066 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:45,067 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796304820_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:45,067 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796304820_0001_00_000001 completed successfully
2016-01-14 18:31:45,067 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796304820_0001_00_000002
2016-01-14 18:31:45,068 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:45,069 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:45,072 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:45,073 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:45,073 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796304820_0001_00_000002, launchTime=1452796305072
2016-01-14 18:31:45,073 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796304820_0001_00_000002
2016-01-14 18:31:45,074 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796304820_0001_1_01_000000_0] to container: [container_1452796304820_0001_00_000002]
2016-01-14 18:31:45,074 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:45,074 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796304820_0001_1_01_000000_0] started. Is using containerId: [container_1452796304820_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:45,074 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=A244E91154794479B2F018E69B105523, taskAttemptId=attempt_1452796304820_0001_1_01_000000_0, startTime=1452796305074, containerId=container_1452796304820_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796304820_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:45,074 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:45,074 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304820_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:45,074 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304820_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:45,075 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796304820_0001_00_000002 given task: attempt_1452796304820_0001_1_01_000000_0
2016-01-14 18:31:45,075 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796304820_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796304820_0001_1_01_000000_0
2016-01-14 18:31:45 Starting to run new task attempt: attempt_1452796304820_0001_1_01_000000_0
2016-01-14 18:31:45,075 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796304820_0001_1_01_000000_0
2016-01-14 18:31:45,075 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,075 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:45,075 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:45,076 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:45,076 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:45,076 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:45,076 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53] every/(1/1) ...lyplatform/ALL_VALUES_ALL, VertexName: A244E91154794479B2F018E69B105523, VertexParallelism: 1, TaskAttemptID:attempt_1452796304820_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=A3067317D9EC4856AD4C276687EEC110, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=7EBDCFE34E83486794DA8904FE8352B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:45,076 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:45,078 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796304820_0001_1_01_000000_0
2016-01-14 18:31:45,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:45,078 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,078 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796304820_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:45,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=A3067317D9EC4856AD4C276687EEC110, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:45,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:45,079 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: A3067317D9EC4856AD4C276687EEC110
2016-01-14 18:31:45,081 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:45,081 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:45,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=7EBDCFE34E83486794DA8904FE8352B9, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:45,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:45,081 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 7EBDCFE34E83486794DA8904FE8352B9
2016-01-14 18:31:45,081 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:45,082 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 7EBDCFE34E83486794DA8904FE8352B9
2016-01-14 18:31:45,084 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:45,084 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: A3067317D9EC4856AD4C276687EEC110
2016-01-14 18:31:45,085 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: A3067317D9EC4856AD4C276687EEC110
2016-01-14 18:31:45,087 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: A244E91154794479B2F018E69B105523, ordinal: 1
2016-01-14 18:31:45,087 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A244E91154794479B2F018E69B105523, mem on start (mb), free: 284, total: 564, max: 672
2016-01-14 18:31:45,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:45,087 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:45,090 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:45,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 7EBDCFE34E83486794DA8904FE8352B9
2016-01-14 18:31:45,090 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:45,090 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:45,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:45,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:45,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:45,090 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:45,091 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:45,091 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 7EBDCFE34E83486794DA8904FE8352B9, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:45,091 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, A3067317D9EC4856AD4C276687EEC110, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:45,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:45,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: A3067317D9EC4856AD4C276687EEC110 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:45,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: A3067317D9EC4856AD4C276687EEC110
2016-01-14 18:31:45,091 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:45,091 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:45,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:45,091 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:45,092 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: A3067317D9EC4856AD4C276687EEC110 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:45,092 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:45,092 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:45,092 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:45,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: A3067317D9EC4856AD4C276687EEC110
2016-01-14 18:31:45,093 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:45,093 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796304820_0001_1_01_000000_0
2016-01-14 18:31:45,094 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796304820_0001_1_00_000000_0_10302, runDuration: 52673, ]
2016-01-14 18:31:45,096 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796304820_0001_1_00_000000_0_10302] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796305096, TimeTaken=0, Rate=0.00 MB/s
2016-01-14 18:31:45,096 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : A3067317D9EC4856AD4C276687EEC110
2016-01-14 18:31:45,096 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:45,096 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:45,096 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [A3067317D9EC4856AD4C276687EEC110] #51 in 1ms
2016-01-14 18:31:45,096 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [A3067317D9EC4856AD4C276687EEC110] #51, status:false, isInterrupted:false
2016-01-14 18:31:45,096 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: FF85E0AB86F64AACB485CF913752208F
2016-01-14 18:31:45,096 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL"], id: 7EBDCFE34E83486794DA8904FE8352B9
2016-01-14 18:31:45,097 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_ALL"]
2016-01-14 18:31:45,097 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] FF85E0AB86F64AACB485CF913752208F, for 1 inputs
2016-01-14 18:31:45,097 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:45,097 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:45,097 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:45,097 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:45,098 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:45,098 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : A3067317D9EC4856AD4C276687EEC110
2016-01-14 18:31:45,098 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:45,098 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: A244E91154794479B2F018E69B105523, all 1 inputs ready in: 00:00:00.001
2016-01-14 18:31:45,098 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6dc71e03; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2b838f43
2016-01-14 18:31:45,099 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:45,099 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:45,099 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:45,099 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:45,100 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796304820_0001_1_01_000000_0
2016-01-14 18:31:45,100 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796304820_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:45,100 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796304820_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:45,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796305098, Process_Duration=2, Process_End_Time=1452796305100, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:45,103 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: A244E91154794479B2F018E69B105523, mem on close (mb), free: 281, total: 564, max: 672
2016-01-14 18:31:45,103 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=A244E91154794479B2F018E69B105523, taskAttemptId=attempt_1452796304820_0001_1_01_000000_0, startTime=1452796305074, finishTime=1452796305103, timeTaken=29, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305098, Process_Duration=2, Process_End_Time=1452796305100, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:45,103 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304820_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:45,104 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796304820_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:45,103 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:45,104 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:TASK_FINISHED]: vertexName=A244E91154794479B2F018E69B105523, taskId=task_1452796304820_0001_1_01_000000, startTime=1452796305074, finishTime=1452796305104, timeTaken=30, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:45,104 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796304820_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:45,104 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:45,104 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:45,103 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=A244E91154794479B2F018E69B105523, index=1
2016-01-14 18:31:45,104 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] : 1
2016-01-14 18:31:45,105 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] because task failed: task_1452796304820_0001_1_01_000000
2016-01-14 18:31:45,105 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:45,105 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:45,105 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523]
2016-01-14 18:31:45,105 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=7EBDCFE34E83486794DA8904FE8352B9, vertexId=vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523]
2016-01-14 18:31:45,106 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: A3067317D9EC4856AD4C276687EEC110
2016-01-14 18:31:45,106 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:45,106 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:45,106 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:VERTEX_FINISHED]: vertexName=A244E91154794479B2F018E69B105523, vertexId=vertex_1452796304820_0001_1_01, initRequestedTime=1452796304981, initedTime=1452796304987, startRequestedTime=1452796304991, startedTime=1452796304991, finishTime=1452796305106, timeTaken=115, status=FAILED, diagnostics=Task failed, taskId=task_1452796304820_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796305074, firstTasksToStart=[ task_1452796304820_0001_1_01_000000 ], lastTaskFinishTime=1452796305103, lastTasksToFinish=[ task_1452796304820_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:45,107 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:45,107 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796304820_0001_00_000002, stoppedTime=1452796305107, exitStatus=0
2016-01-14 18:31:45,107 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:45,106 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:45,107 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796304820_0001_1_01_000000_0 failed: true container: container_1452796304820_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:45,107 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:45,108 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:45,108 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796304820_0001_1
2016-01-14 18:31:45,108 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 7EBDCFE34E83486794DA8904FE8352B9 for vertex: vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523]
2016-01-14 18:31:45,107 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:45,108 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:45,108 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796304820_0001_1_00 [A3067317D9EC4856AD4C276687EEC110]
2016-01-14 18:31:45,108 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:45,108 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:45,108 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=A244E91154794479B2F018E69B105523, sourceVertex=A3067317D9EC4856AD4C276687EEC110
2016-01-14 18:31:45,108 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=A244E91154794479B2F018E69B105523, sourceVertex=7EBDCFE34E83486794DA8904FE8352B9
2016-01-14 18:31:45,109 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796304820_0001_00_000002
2016-01-14 18:31:45,109 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796304820_0001_00_000002
2016-01-14 18:31:45,109 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796304820_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:45,109 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796304820_0001_00_000002 completed but with errors
2016-01-14 18:31:45,109 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796304820_0001_1, queueSize=0
2016-01-14 18:31:45,110 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:45,110 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796304820_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796304820_0001_1, startTime=1452796304980, finishTime=1452796305108, timeTaken=128, status=FAILED, diagnostics=Vertex failed, vertexName=A244E91154794479B2F018E69B105523, vertexId=vertex_1452796304820_0001_1_01, diagnostics=[Task failed, taskId=task_1452796304820_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796304820_0001_1_01 [A244E91154794479B2F018E69B105523] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591396864, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305051, Process_Duration=3, Process_End_Time=1452796305054, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:45,111 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796304820_0001_1 finished with state: FAILED
2016-01-14 18:31:45,112 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796304820_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:45,112 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796304820_0001_00_000002
2016-01-14 18:31:45,112 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796304820_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:45,112 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796304820_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:45 Completed Dag: dag_1452796304820_0001_1
2016-01-14 18:31:45,113 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796304820_0001_1, dagState=FAILED
2016-01-14 18:31:45,113 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796304820_0001_1_post
2016-01-14 18:31:45,114 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,114 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:45,114 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796304820_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:45,117 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...lyplatform/ALL_VALUES_ALL, applicationId=application_1452796304820_0001
2016-01-14 18:31:45,118 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:45,118 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:45,118 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:45,118 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:45,118 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:45,119 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:45,119 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:45,119 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:45,119 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:45,119 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:45,119 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:45,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:45,119 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:45,123 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/80D1AFFF2E934E8EA36A772EEFDE107A/F3ABB5D7EB534D279F1C332CC41D5C53/.tez/application_1452796304820_0001
2016-01-14 18:31:45,123 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:45,124 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...lyplatform/ALL_VALUES_ALL
2016-01-14 18:31:45,125 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:45,125 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.323
2016-01-14 18:31:45,125 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_ALL
2016-01-14 18:31:45,146 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:45,146 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:45,175 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 18:31:45,183 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 18:31:45,184 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:45,184 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:45,184 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:45,198 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:45,202 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE
2016-01-14 18:31:45,243 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:45,243 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:45,243 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE"]
2016-01-14 18:31:45,243 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:45,243 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:45,243 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:45,244 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 18:31:45,244 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:45,256 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:45,260 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:45,260 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:45,260 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:45,269 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003/.tez/application_1452796305260_0001 doesn't exist and is created
2016-01-14 18:31:45,282 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:45,282 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:45,283 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003/.tez/application_1452796305260_0001
2016-01-14 18:31:45,284 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796305260_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:45,296 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:45,297 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:45,298 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:45,298 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:45,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003/.tez/application_1452796305260_0001/localmode-log-dir
2016-01-14 18:31:45,298 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003/.tez/application_1452796305260_0001/localmode-log-dir/history.txt.appattempt_1452796305260_0001_000000, maxErrors=10
2016-01-14 18:31:45,298 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:45,298 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796305260_0001_000000, appSubmitTime=1452796305284, launchTime=1452796305284
2016-01-14 18:31:45,302 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:45,303 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:45,303 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:45,303 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:45,304 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:52243
2016-01-14 18:31:45,305 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:45,307 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796305260_0001_000000, startTime=1452796305307
2016-01-14 18:31:45,307 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:45,307 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:45,383 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:45,383 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:45,383 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1452796305260_0001, dagName=[281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 18:31:45,407 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 18:31:45,409 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796305260_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003/.tez/application_1452796305260_0001/localmode-log-dir/dag_1452796305260_0001_1.dot
2016-01-14 18:31:45,409 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003/.tez/application_1452796305260_0001/localmode-log-dir/dag_1452796305260_0001_1-tez-dag.pb.txt
2016-01-14 18:31:45,421 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796305260_0001_1
2016-01-14 18:31:45,421 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,421 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 18:31:45 Running Dag: dag_1452796305260_0001_1
2016-01-14 18:31:45,431 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796305260_0001_1, submitTime=1452796305407
2016-01-14 18:31:45,431 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:45,432 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E8BD54572EDB44E79AEE75C601666F72
2016-01-14 18:31:45,432 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:45,432 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:45,432 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796305260_0001_1, initTime=1452796305431
2016-01-14 18:31:45,432 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796305260_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:45,433 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:DAG_STARTED]: dagID=dag_1452796305260_0001_1, startTime=1452796305432
2016-01-14 18:31:45,433 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:45,433 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796305260_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:45,433 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C7614C85EA8D407AA2D60CBAE39D3880 : {F9861B61040E4A109B3DC1E46A000D93={InputName=F9861B61040E4A109B3DC1E46A000D93}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:45,433 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: F9861B61040E4A109B3DC1E46A000D93, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:45,433 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880]
2016-01-14 18:31:45,433 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880]
2016-01-14 18:31:45,433 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880]
2016-01-14 18:31:45,434 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880]. Starting root input initializers: 1
2016-01-14 18:31:45,434 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:45,434 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:45,434 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72]
2016-01-14 18:31:45,434 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1452796305260_0001, dagName=[281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003] every/(1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 18:31:45,438 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:45,438 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72]
2016-01-14 18:31:45,438 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72]
2016-01-14 18:31:45,438 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72]
2016-01-14 18:31:45,439 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=102A54DA13DA4B159BF7C1153CD25F1A, vertexId=vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:45,439 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=102A54DA13DA4B159BF7C1153CD25F1A, vertexId=vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72]
2016-01-14 18:31:45,441 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=102A54DA13DA4B159BF7C1153CD25F1A, vertexName=E8BD54572EDB44E79AEE75C601666F72, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:45,441 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=102A54DA13DA4B159BF7C1153CD25F1A, vertexId=vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72]
2016-01-14 18:31:45,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:45,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E8BD54572EDB44E79AEE75C601666F72, vertexId=vertex_1452796305260_0001_1_01, initRequestedTime=1452796305434, initedTime=1452796305442, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:45,443 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:45,443 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: F9861B61040E4A109B3DC1E46A000D93 on vertex vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880]
2016-01-14 18:31:45,445 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:45,445 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input F9861B61040E4A109B3DC1E46A000D93 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:45,445 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:45,446 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:45,447 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:45,447 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: F9861B61040E4A109B3DC1E46A000D93 on vertex vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880]
2016-01-14 18:31:45,448 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:45,448 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {F9861B61040E4A109B3DC1E46A000D93=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:45,448 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880] parallelism set to 1
2016-01-14 18:31:45,448 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:45,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796305260_0001_1_00, eventCount=1
2016-01-14 18:31:45,449 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:45,449 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C7614C85EA8D407AA2D60CBAE39D3880, vertexId=vertex_1452796305260_0001_1_00, initRequestedTime=1452796305433, initedTime=1452796305449, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:45,449 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880] with distanceFromRoot: 0
2016-01-14 18:31:45,450 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:45,450 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C7614C85EA8D407AA2D60CBAE39D3880
2016-01-14 18:31:45,450 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880]
2016-01-14 18:31:45,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:45,451 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796305260_0001_1_00, startRequestedTime=1452796305450, startedTime=1452796305450
2016-01-14 18:31:45,451 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:45,451 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:45,452 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:TASK_STARTED]: vertexName=C7614C85EA8D407AA2D60CBAE39D3880, taskId=task_1452796305260_0001_1_00_000000, scheduledTime=1452796305451, launchTime=1452796305451
2016-01-14 18:31:45,452 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305260_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:45,452 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796305260_0001_1_00 for vertex: vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] numStartedSources: 1 numSources: 1
2016-01-14 18:31:45,452 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] with distanceFromRoot: 1
2016-01-14 18:31:45,452 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880]
2016-01-14 18:31:45,453 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C7614C85EA8D407AA2D60CBAE39D3880 in vertex: E8BD54572EDB44E79AEE75C601666F72
2016-01-14 18:31:45,453 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E8BD54572EDB44E79AEE75C601666F72 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:45,453 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:45,453 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796305260_0001, with dag id: dag_1452796305260_0001_1
2016-01-14 18:31:45,454 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796305260_0001_1_01, startRequestedTime=1452796305453, startedTime=1452796305453
2016-01-14 18:31:45,454 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:45,454 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: C7614C85EA8D407AA2D60CBAE39D3880, VertexParallelism: 1, TaskAttemptID:attempt_1452796305260_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9861B61040E4A109B3DC1E46A000D93, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E8BD54572EDB44E79AEE75C601666F72, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:45,455 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305260_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:45,457 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:45,457 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:45,458 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:45,458 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:45,458 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796305260_0001_00_000001
2016-01-14 18:31:45,458 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:45,460 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796305260_0001_00_000001
2016-01-14 18:31:45,460 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:45,461 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:45,461 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:45,461 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796305260_0001_00_000001, launchTime=1452796305461
2016-01-14 18:31:45,462 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796305260_0001_00_000001
2016-01-14 18:31:45,462 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796305260_0001_1_00_000000_0] to container: [container_1452796305260_0001_00_000001]
2016-01-14 18:31:45,462 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:45,463 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796305260_0001_1_00_000000_0] started. Is using containerId: [container_1452796305260_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:45,463 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C7614C85EA8D407AA2D60CBAE39D3880, taskAttemptId=attempt_1452796305260_0001_1_00_000000_0, startTime=1452796305463, containerId=container_1452796305260_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796305260_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:45,463 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:45,463 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796305260_0001_00_000001 given task: attempt_1452796305260_0001_1_00_000000_0
2016-01-14 18:31:45,463 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:45,463 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:45,463 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305260_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:45,464 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305260_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:45,463 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796305260_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796305260_0001_1_00_000000_0
2016-01-14 18:31:45 Starting to run new task attempt: attempt_1452796305260_0001_1_00_000000_0
2016-01-14 18:31:45,464 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796305260_0001_1_00_000000_0
2016-01-14 18:31:45,464 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,465 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:45,465 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:45,465 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:45,465 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:45,465 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:45,465 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: C7614C85EA8D407AA2D60CBAE39D3880, VertexParallelism: 1, TaskAttemptID:attempt_1452796305260_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=F9861B61040E4A109B3DC1E46A000D93, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E8BD54572EDB44E79AEE75C601666F72, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:45,465 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:45,466 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796305260_0001_1_00_000000_0
2016-01-14 18:31:45,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:45,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,467 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796305260_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:45,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=F9861B61040E4A109B3DC1E46A000D93, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:45,467 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:45,468 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: F9861B61040E4A109B3DC1E46A000D93
2016-01-14 18:31:45,468 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: F9861B61040E4A109B3DC1E46A000D93
2016-01-14 18:31:45,470 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:45,471 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: F9861B61040E4A109B3DC1E46A000D93
2016-01-14 18:31:45,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: F9861B61040E4A109B3DC1E46A000D93
2016-01-14 18:31:45,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E8BD54572EDB44E79AEE75C601666F72, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:45,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:45,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E8BD54572EDB44E79AEE75C601666F72
2016-01-14 18:31:45,474 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:45,475 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:45,475 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E8BD54572EDB44E79AEE75C601666F72
2016-01-14 18:31:45,475 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E8BD54572EDB44E79AEE75C601666F72
2016-01-14 18:31:45,477 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C7614C85EA8D407AA2D60CBAE39D3880, ordinal: 0
2016-01-14 18:31:45,477 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7614C85EA8D407AA2D60CBAE39D3880, mem on start (mb), free: 193, total: 564, max: 672
2016-01-14 18:31:45,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:45,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:45,477 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:45,477 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:45,477 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:45,477 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:45,477 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:45,478 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:45,478 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:45,478 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, F9861B61040E4A109B3DC1E46A000D93, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:45,478 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E8BD54572EDB44E79AEE75C601666F72, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:45,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:45,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: F9861B61040E4A109B3DC1E46A000D93 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:45,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: F9861B61040E4A109B3DC1E46A000D93
2016-01-14 18:31:45,479 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:45,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: F9861B61040E4A109B3DC1E46A000D93
2016-01-14 18:31:45,478 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:45,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:45,479 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:45,480 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796305260_0001_1_00_000000_0
2016-01-14 18:31:45,483 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:45,483 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: F9861B61040E4A109B3DC1E46A000D93
2016-01-14 18:31:45,484 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: F93CA2AAAB75484FAA1213289577723F
2016-01-14 18:31:45,484 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] F93CA2AAAB75484FAA1213289577723F
2016-01-14 18:31:45,485 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@42017816; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@6e730e34; comparator=cascading.tuple.hadoop.util.TupleComparator@43a9b092; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:45,485 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:45,486 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:45,486 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@67b2c25f
2016-01-14 18:31:45,486 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:45,487 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:45,487 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@67b2c25f
2016-01-14 18:31:45,498 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:45,498 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:45,511 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:45,511 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:45,511 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:45,511 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:45,511 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:45,512 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:45,512 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:45,513 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C7614C85EA8D407AA2D60CBAE39D3880, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:45,515 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796305260_0001_1_00_000000_0
2016-01-14 18:31:45,515 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7614C85EA8D407AA2D60CBAE39D3880, mem on close (mb), free: 289, total: 565, max: 672
2016-01-14 18:31:45,515 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:45,515 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:45,515 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:45,515 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:45,517 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:45 Completed running task attempt: attempt_1452796305260_0001_1_00_000000_0
2016-01-14 18:31:45,520 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796305260_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:45,520 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796305513, Process_Duration=2, Process_End_Time=1452796305515, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:45,520 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796305260_0001_1_00, eventCount=1
2016-01-14 18:31:45,521 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:45,521 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C7614C85EA8D407AA2D60CBAE39D3880, taskAttemptId=attempt_1452796305260_0001_1_00_000000_0, startTime=1452796305463, finishTime=1452796305521, timeTaken=58, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305513, Process_Duration=2, Process_End_Time=1452796305515, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:45,521 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305260_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:45,521 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:45,521 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796305260_0001_1_00_000000_0
2016-01-14 18:31:45,522 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:45,522 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:TASK_FINISHED]: vertexName=C7614C85EA8D407AA2D60CBAE39D3880, taskId=task_1452796305260_0001_1_00_000000, startTime=1452796305463, finishTime=1452796305522, timeTaken=59, status=SUCCEEDED, successfulAttemptID=attempt_1452796305260_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305513, Process_Duration=2, Process_End_Time=1452796305515, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:45,522 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305260_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:45,522 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880] : 1
2016-01-14 18:31:45,522 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880]
2016-01-14 18:31:45,523 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:45,523 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:VERTEX_FINISHED]: vertexName=C7614C85EA8D407AA2D60CBAE39D3880, vertexId=vertex_1452796305260_0001_1_00, initRequestedTime=1452796305433, initedTime=1452796305449, startRequestedTime=1452796305450, startedTime=1452796305450, finishTime=1452796305522, timeTaken=72, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305513, Process_Duration=2, Process_End_Time=1452796305515, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796305463, firstTasksToStart=[ task_1452796305260_0001_1_00_000000 ], lastTaskFinishTime=1452796305521, lastTasksToFinish=[ task_1452796305260_0001_1_00_000000 ], minTaskDuration=58, maxTaskDuration=58, avgTaskDuration=58.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796305260_0001_1_00_000000 ], longestDurationTasks=[ task_1452796305260_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:45,523 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:45,523 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:45,523 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796305260_0001_00_000001, stoppedTime=1452796305523, exitStatus=0
2016-01-14 18:31:45,524 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:45,524 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] attempt: attempt_1452796305260_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:45,523 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796305260_0001_00_000001
2016-01-14 18:31:45,524 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E8BD54572EDB44E79AEE75C601666F72
2016-01-14 18:31:45,524 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72]
2016-01-14 18:31:45,524 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:45,523 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:45,525 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:45,525 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796305260_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:45,525 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:TASK_STARTED]: vertexName=E8BD54572EDB44E79AEE75C601666F72, taskId=task_1452796305260_0001_1_01_000000, scheduledTime=1452796305525, launchTime=1452796305525
2016-01-14 18:31:45,525 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305260_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:45,525 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:45,526 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72]
2016-01-14 18:31:45,526 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: E8BD54572EDB44E79AEE75C601666F72, VertexParallelism: 1, TaskAttemptID:attempt_1452796305260_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C7614C85EA8D407AA2D60CBAE39D3880, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=102A54DA13DA4B159BF7C1153CD25F1A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:45,526 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305260_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:45,526 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:45,526 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:45,527 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796305260_0001_00_000002
2016-01-14 18:31:45,527 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:45,528 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796305260_0001_00_000002
2016-01-14 18:31:45,529 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:45,529 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796305260_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:45,529 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796305260_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:45,529 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796305260_0001_00_000001, Exiting
2016-01-14 18:31:45,529 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796305260_0001_00_000001
2016-01-14 18:31:45,529 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796305260_0001_00_000001
2016-01-14 18:31:45,530 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:45,531 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:45,531 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796305260_0001_00_000002, launchTime=1452796305531
2016-01-14 18:31:45,532 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796305260_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:45,532 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796305260_0001_00_000001 completed successfully
2016-01-14 18:31:45,532 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:45,532 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796305260_0001_00_000002
2016-01-14 18:31:45,533 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796305260_0001_1_01_000000_0] to container: [container_1452796305260_0001_00_000002]
2016-01-14 18:31:45,533 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:45,533 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796305260_0001_1_01_000000_0] started. Is using containerId: [container_1452796305260_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:45,534 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E8BD54572EDB44E79AEE75C601666F72, taskAttemptId=attempt_1452796305260_0001_1_01_000000_0, startTime=1452796305533, containerId=container_1452796305260_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796305260_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:45,534 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305260_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:45,534 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305260_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:45,534 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:45,534 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796305260_0001_00_000002 given task: attempt_1452796305260_0001_1_01_000000_0
2016-01-14 18:31:45,534 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796305260_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796305260_0001_1_01_000000_0
2016-01-14 18:31:45 Starting to run new task attempt: attempt_1452796305260_0001_1_01_000000_0
2016-01-14 18:31:45,535 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796305260_0001_1_01_000000_0
2016-01-14 18:31:45,535 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,535 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:45,535 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:45,535 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:45,535 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:45,535 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:45,535 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003] every/(1/1) ...atform/ALL_VALUES_REPLACE, VertexName: E8BD54572EDB44E79AEE75C601666F72, VertexParallelism: 1, TaskAttemptID:attempt_1452796305260_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C7614C85EA8D407AA2D60CBAE39D3880, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=102A54DA13DA4B159BF7C1153CD25F1A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:45,536 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:45,536 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796305260_0001_1_01_000000_0
2016-01-14 18:31:45,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:45,536 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,536 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796305260_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:45,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C7614C85EA8D407AA2D60CBAE39D3880, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:45,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:45,537 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C7614C85EA8D407AA2D60CBAE39D3880
2016-01-14 18:31:45,539 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:45,540 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:45,540 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C7614C85EA8D407AA2D60CBAE39D3880
2016-01-14 18:31:45,540 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:45,540 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C7614C85EA8D407AA2D60CBAE39D3880
2016-01-14 18:31:45,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=102A54DA13DA4B159BF7C1153CD25F1A, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:45,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:45,542 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 102A54DA13DA4B159BF7C1153CD25F1A
2016-01-14 18:31:45,543 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:45,543 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 102A54DA13DA4B159BF7C1153CD25F1A
2016-01-14 18:31:45,544 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E8BD54572EDB44E79AEE75C601666F72, ordinal: 1
2016-01-14 18:31:45,544 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8BD54572EDB44E79AEE75C601666F72, mem on start (mb), free: 284, total: 565, max: 672
2016-01-14 18:31:45,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:45,544 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:45,553 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:45,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 102A54DA13DA4B159BF7C1153CD25F1A
2016-01-14 18:31:45,553 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:45,553 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:45,553 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:45,553 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:45,553 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:45,553 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:45,554 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:45,554 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C7614C85EA8D407AA2D60CBAE39D3880, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:45,554 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 102A54DA13DA4B159BF7C1153CD25F1A, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:45,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:45,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C7614C85EA8D407AA2D60CBAE39D3880 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:45,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C7614C85EA8D407AA2D60CBAE39D3880
2016-01-14 18:31:45,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:45,554 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:45,554 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:45,555 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:45,555 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C7614C85EA8D407AA2D60CBAE39D3880 with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:45,555 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:45,555 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:45,555 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:45,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C7614C85EA8D407AA2D60CBAE39D3880
2016-01-14 18:31:45,556 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:45,556 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796305260_0001_1_01_000000_0
2016-01-14 18:31:45,557 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796305260_0001_1_00_000000_0_10309, runDuration: 45243, ]
2016-01-14 18:31:45,559 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796305260_0001_1_00_000000_0_10309] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796305559, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:45,559 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C7614C85EA8D407AA2D60CBAE39D3880
2016-01-14 18:31:45,559 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:45,560 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:45,560 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C7614C85EA8D407AA2D60CBAE39D3880] #52 in 2ms
2016-01-14 18:31:45,560 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C7614C85EA8D407AA2D60CBAE39D3880] #52, status:false, isInterrupted:false
2016-01-14 18:31:45,560 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:45,560 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:45,561 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:45,561 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:45,561 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:45,561 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C7614C85EA8D407AA2D60CBAE39D3880
2016-01-14 18:31:45,561 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:45,560 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: F93CA2AAAB75484FAA1213289577723F
2016-01-14 18:31:45,562 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE"], id: 102A54DA13DA4B159BF7C1153CD25F1A
2016-01-14 18:31:45,562 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_REPLACE"]
2016-01-14 18:31:45,562 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] F93CA2AAAB75484FAA1213289577723F, for 1 inputs
2016-01-14 18:31:45,562 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E8BD54572EDB44E79AEE75C601666F72, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:45,563 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@23ca6e05; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3efd1c9e
2016-01-14 18:31:45,563 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:45,566 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:45,567 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:45,568 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:45,568 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796305260_0001_1_01_000000_0
2016-01-14 18:31:45,568 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796305260_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:45,569 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796305260_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:45,571 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796305562, Process_Duration=7, Process_End_Time=1452796305569, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:45,571 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:45,571 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E8BD54572EDB44E79AEE75C601666F72, taskAttemptId=attempt_1452796305260_0001_1_01_000000_0, startTime=1452796305533, finishTime=1452796305571, timeTaken=38, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305562, Process_Duration=7, Process_End_Time=1452796305569, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:45,572 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305260_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:45,572 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796305260_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:45,572 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:45,572 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E8BD54572EDB44E79AEE75C601666F72, mem on close (mb), free: 280, total: 565, max: 672
2016-01-14 18:31:45,572 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:TASK_FINISHED]: vertexName=E8BD54572EDB44E79AEE75C601666F72, taskId=task_1452796305260_0001_1_01_000000, startTime=1452796305533, finishTime=1452796305572, timeTaken=39, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:45,573 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305260_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:45,573 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] : 1
2016-01-14 18:31:45,573 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] because task failed: task_1452796305260_0001_1_01_000000
2016-01-14 18:31:45,573 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:45,573 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:45,573 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72]
2016-01-14 18:31:45,573 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=102A54DA13DA4B159BF7C1153CD25F1A, vertexId=vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72]
2016-01-14 18:31:45,573 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E8BD54572EDB44E79AEE75C601666F72, index=1
2016-01-14 18:31:45,573 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:45,574 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C7614C85EA8D407AA2D60CBAE39D3880
2016-01-14 18:31:45,574 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:45,574 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:45,575 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:45,575 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:45,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:VERTEX_FINISHED]: vertexName=E8BD54572EDB44E79AEE75C601666F72, vertexId=vertex_1452796305260_0001_1_01, initRequestedTime=1452796305434, initedTime=1452796305442, startRequestedTime=1452796305453, startedTime=1452796305453, finishTime=1452796305574, timeTaken=121, status=FAILED, diagnostics=Task failed, taskId=task_1452796305260_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796305533, firstTasksToStart=[ task_1452796305260_0001_1_01_000000 ], lastTaskFinishTime=1452796305571, lastTasksToFinish=[ task_1452796305260_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:45,575 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:45,575 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796305260_0001_00_000002, stoppedTime=1452796305575, exitStatus=0
2016-01-14 18:31:45,575 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:45,575 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796305260_0001_1_01_000000_0 failed: true container: container_1452796305260_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:45,575 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:45,576 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:45,576 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796305260_0001_1
2016-01-14 18:31:45,576 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796305260_0001_1_00 [C7614C85EA8D407AA2D60CBAE39D3880]
2016-01-14 18:31:45,576 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 102A54DA13DA4B159BF7C1153CD25F1A for vertex: vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72]
2016-01-14 18:31:45,575 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:45,575 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:45,577 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796305260_0001_1, queueSize=0
2016-01-14 18:31:45,577 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:45,577 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E8BD54572EDB44E79AEE75C601666F72, sourceVertex=C7614C85EA8D407AA2D60CBAE39D3880
2016-01-14 18:31:45,578 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E8BD54572EDB44E79AEE75C601666F72, sourceVertex=102A54DA13DA4B159BF7C1153CD25F1A
2016-01-14 18:31:45,578 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796305260_0001_00_000002
2016-01-14 18:31:45,578 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796305260_0001_00_000002
2016-01-14 18:31:45,577 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305260_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796305260_0001_1, startTime=1452796305432, finishTime=1452796305576, timeTaken=144, status=FAILED, diagnostics=Vertex failed, vertexName=E8BD54572EDB44E79AEE75C601666F72, vertexId=vertex_1452796305260_0001_1_01, diagnostics=[Task failed, taskId=task_1452796305260_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796305260_0001_1_01 [E8BD54572EDB44E79AEE75C601666F72] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=10, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305513, Process_Duration=2, Process_End_Time=1452796305515, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:45,578 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796305260_0001_1 finished with state: FAILED
2016-01-14 18:31:45,578 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796305260_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:45,577 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:45,579 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796305260_0001_00_000002
2016-01-14 18:31:45,578 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796305260_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:45,579 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796305260_0001_00_000002 completed but with errors
2016-01-14 18:31:45 Completed Dag: dag_1452796305260_0001_1
2016-01-14 18:31:45,583 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796305260_0001_1, dagState=FAILED
2016-01-14 18:31:45,583 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796305260_0001_1_post
2016-01-14 18:31:45,583 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,583 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:45,583 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...atform/ALL_VALUES_REPLACE, applicationId=application_1452796305260_0001
2016-01-14 18:31:45,583 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:45,583 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796305260_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:45,583 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305260_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:45,583 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305260_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:45,585 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:45,585 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:45,585 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:45,585 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:45,585 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:45,586 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:45,586 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:45,586 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:45,586 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:45,586 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:45,586 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:45,587 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:45,587 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:45,591 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/281D3EDBD65F4930B326D81026CA2088/F60FE610EBE54BB889BEC3AE2F920003/.tez/application_1452796305260_0001
2016-01-14 18:31:45,591 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:45,591 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...atform/ALL_VALUES_REPLACE
2016-01-14 18:31:45,592 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:45,592 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.349
2016-01-14 18:31:45,592 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_REPLACE
2016-01-14 18:31:45,620 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:45,620 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:45,658 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 18:31:45,661 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.017
2016-01-14 18:31:45,661 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:45,661 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:45,661 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:45,683 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:45,689 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP
2016-01-14 18:31:45,730 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:45,730 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:45,730 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP"]
2016-01-14 18:31:45,730 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:45,730 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:45,731 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:45,731 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 18:31:45,731 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:45,746 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:45,751 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:45,751 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:45,751 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:45,760 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5/.tez/application_1452796305751_0001 doesn't exist and is created
2016-01-14 18:31:45,773 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:45,774 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:45,774 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5/.tez/application_1452796305751_0001
2016-01-14 18:31:45,775 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796305751_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:45,785 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:45,786 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:45,786 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:45,786 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:45,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5/.tez/application_1452796305751_0001/localmode-log-dir
2016-01-14 18:31:45,787 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5/.tez/application_1452796305751_0001/localmode-log-dir/history.txt.appattempt_1452796305751_0001_000000, maxErrors=10
2016-01-14 18:31:45,787 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:45,787 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796305751_0001_000000, appSubmitTime=1452796305774, launchTime=1452796305775
2016-01-14 18:31:45,791 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:36578
2016-01-14 18:31:45,793 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:45,793 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:45,793 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:45,794 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:45,795 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:45,795 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796305751_0001_000000, startTime=1452796305795
2016-01-14 18:31:45,796 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:45,796 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:45,874 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:45,874 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:45,874 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1452796305751_0001, dagName=[FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 18:31:45,891 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 18:31:45,893 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796305751_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5/.tez/application_1452796305751_0001/localmode-log-dir/dag_1452796305751_0001_1.dot
2016-01-14 18:31:45,893 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5/.tez/application_1452796305751_0001/localmode-log-dir/dag_1452796305751_0001_1-tez-dag.pb.txt
2016-01-14 18:31:45,899 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796305751_0001_1
2016-01-14 18:31:45,899 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,900 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 18:31:45 Running Dag: dag_1452796305751_0001_1
2016-01-14 18:31:45,909 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796305751_0001_1, submitTime=1452796305891
2016-01-14 18:31:45,909 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:45,910 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex BEFF6B05109E4AA594A5BBF15B6C939D
2016-01-14 18:31:45,910 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:45,910 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796305751_0001_1, initTime=1452796305909
2016-01-14 18:31:45,910 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796305751_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:45,910 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:45,912 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:DAG_STARTED]: dagID=dag_1452796305751_0001_1, startTime=1452796305912
2016-01-14 18:31:45,912 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:45,912 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1452796305751_0001, dagName=[FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5] every/(1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 18:31:45,912 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:45,912 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796305751_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:45,912 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: AF17A5BE952547B0A81574220178C57D : {E3F1ED42EF1B438E91CF3F13DC78970A={InputName=E3F1ED42EF1B438E91CF3F13DC78970A}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:45,913 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: E3F1ED42EF1B438E91CF3F13DC78970A, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:45,913 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D]
2016-01-14 18:31:45,913 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D]
2016-01-14 18:31:45,913 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D]
2016-01-14 18:31:45,913 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D]. Starting root input initializers: 1
2016-01-14 18:31:45,913 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:45,914 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D]
2016-01-14 18:31:45,917 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:45,917 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D]
2016-01-14 18:31:45,917 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D]
2016-01-14 18:31:45,917 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D]
2016-01-14 18:31:45,917 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=02C6652ECA764D2893967E498B66FE4F, vertexId=vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:45,917 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=02C6652ECA764D2893967E498B66FE4F, vertexId=vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D]
2016-01-14 18:31:45,918 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: E3F1ED42EF1B438E91CF3F13DC78970A on vertex vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D]
2016-01-14 18:31:45,919 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=02C6652ECA764D2893967E498B66FE4F, vertexName=BEFF6B05109E4AA594A5BBF15B6C939D, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:45,919 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=02C6652ECA764D2893967E498B66FE4F, vertexId=vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D]
2016-01-14 18:31:45,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:45,920 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:VERTEX_INITIALIZED]: vertexName=BEFF6B05109E4AA594A5BBF15B6C939D, vertexId=vertex_1452796305751_0001_1_01, initRequestedTime=1452796305914, initedTime=1452796305920, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:45,921 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:45,921 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:45,921 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input E3F1ED42EF1B438E91CF3F13DC78970A asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:45,922 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:45,922 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:45,923 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:45,923 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: E3F1ED42EF1B438E91CF3F13DC78970A on vertex vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D]
2016-01-14 18:31:45,923 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:45,923 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {E3F1ED42EF1B438E91CF3F13DC78970A=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:45,924 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D] parallelism set to 1
2016-01-14 18:31:45,924 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:45,924 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796305751_0001_1_00, eventCount=1
2016-01-14 18:31:45,924 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:45,925 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:VERTEX_INITIALIZED]: vertexName=AF17A5BE952547B0A81574220178C57D, vertexId=vertex_1452796305751_0001_1_00, initRequestedTime=1452796305912, initedTime=1452796305924, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:45,925 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D] with distanceFromRoot: 0
2016-01-14 18:31:45,925 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:45,925 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in AF17A5BE952547B0A81574220178C57D
2016-01-14 18:31:45,925 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D]
2016-01-14 18:31:45,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:45,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796305751_0001_1_00, startRequestedTime=1452796305925, startedTime=1452796305925
2016-01-14 18:31:45,926 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:45,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:45,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:TASK_STARTED]: vertexName=AF17A5BE952547B0A81574220178C57D, taskId=task_1452796305751_0001_1_00_000000, scheduledTime=1452796305926, launchTime=1452796305926
2016-01-14 18:31:45,927 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305751_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:45,927 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796305751_0001_1_00 for vertex: vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] numStartedSources: 1 numSources: 1
2016-01-14 18:31:45,927 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] with distanceFromRoot: 1
2016-01-14 18:31:45,927 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D]
2016-01-14 18:31:45,927 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: AF17A5BE952547B0A81574220178C57D in vertex: BEFF6B05109E4AA594A5BBF15B6C939D
2016-01-14 18:31:45,928 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: BEFF6B05109E4AA594A5BBF15B6C939D with 1 source tasks and 1 pending tasks
2016-01-14 18:31:45,928 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:45,928 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796305751_0001_1_01, startRequestedTime=1452796305927, startedTime=1452796305927
2016-01-14 18:31:45,928 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:45,928 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: AF17A5BE952547B0A81574220178C57D, VertexParallelism: 1, TaskAttemptID:attempt_1452796305751_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E3F1ED42EF1B438E91CF3F13DC78970A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BEFF6B05109E4AA594A5BBF15B6C939D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:45,929 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305751_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:45,929 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:45,929 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:45,930 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:45,930 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:45,930 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796305751_0001_00_000001
2016-01-14 18:31:45,930 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:45,931 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796305751_0001, with dag id: dag_1452796305751_0001_1
2016-01-14 18:31:45,931 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796305751_0001_00_000001
2016-01-14 18:31:45,931 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:45,933 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:45,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:45,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796305751_0001_00_000001, launchTime=1452796305933
2016-01-14 18:31:45,934 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796305751_0001_00_000001
2016-01-14 18:31:45,934 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796305751_0001_1_00_000000_0] to container: [container_1452796305751_0001_00_000001]
2016-01-14 18:31:45,935 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:45,935 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796305751_0001_00_000001 given task: attempt_1452796305751_0001_1_00_000000_0
2016-01-14 18:31:45,935 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796305751_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796305751_0001_1_00_000000_0
2016-01-14 18:31:45,935 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796305751_0001_1_00_000000_0] started. Is using containerId: [container_1452796305751_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:45,935 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=AF17A5BE952547B0A81574220178C57D, taskAttemptId=attempt_1452796305751_0001_1_00_000000_0, startTime=1452796305935, containerId=container_1452796305751_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796305751_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:45,935 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305751_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:45 Starting to run new task attempt: attempt_1452796305751_0001_1_00_000000_0
2016-01-14 18:31:45,935 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796305751_0001_1_00_000000_0
2016-01-14 18:31:45,935 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305751_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:45,936 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,935 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:45,936 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:45,936 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:45,936 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:45,936 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:45,936 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:45,936 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: AF17A5BE952547B0A81574220178C57D, VertexParallelism: 1, TaskAttemptID:attempt_1452796305751_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=E3F1ED42EF1B438E91CF3F13DC78970A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=BEFF6B05109E4AA594A5BBF15B6C939D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:45,936 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:45,937 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796305751_0001_1_00_000000_0
2016-01-14 18:31:45,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:45,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=E3F1ED42EF1B438E91CF3F13DC78970A, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:45,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:45,938 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796305751_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:45,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=BEFF6B05109E4AA594A5BBF15B6C939D, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:45,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:45,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: BEFF6B05109E4AA594A5BBF15B6C939D
2016-01-14 18:31:45,940 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:45,940 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: BEFF6B05109E4AA594A5BBF15B6C939D
2016-01-14 18:31:45,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: BEFF6B05109E4AA594A5BBF15B6C939D
2016-01-14 18:31:45,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: E3F1ED42EF1B438E91CF3F13DC78970A
2016-01-14 18:31:45,941 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: E3F1ED42EF1B438E91CF3F13DC78970A
2016-01-14 18:31:45,942 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:45,943 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:45,943 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: E3F1ED42EF1B438E91CF3F13DC78970A
2016-01-14 18:31:45,943 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: E3F1ED42EF1B438E91CF3F13DC78970A
2016-01-14 18:31:45,944 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: AF17A5BE952547B0A81574220178C57D, ordinal: 0
2016-01-14 18:31:45,944 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AF17A5BE952547B0A81574220178C57D, mem on start (mb), free: 192, total: 565, max: 672
2016-01-14 18:31:45,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:45,944 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:45,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:45,945 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:45,945 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:45,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:45,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:45,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:45,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:45,945 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:45,945 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, BEFF6B05109E4AA594A5BBF15B6C939D, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:45,946 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, E3F1ED42EF1B438E91CF3F13DC78970A, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:45,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:45,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: E3F1ED42EF1B438E91CF3F13DC78970A being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:45,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:45,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:45,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: E3F1ED42EF1B438E91CF3F13DC78970A
2016-01-14 18:31:45,946 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:45,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: E3F1ED42EF1B438E91CF3F13DC78970A
2016-01-14 18:31:45,946 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:45,947 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796305751_0001_1_00_000000_0
2016-01-14 18:31:45,947 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:45,947 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@45cee553
2016-01-14 18:31:45,948 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@45cee553
2016-01-14 18:31:45,948 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:45,949 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:45,950 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:45,950 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: E3F1ED42EF1B438E91CF3F13DC78970A
2016-01-14 18:31:45,950 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: CFB11C367CDB4D6C992AEDD08213B145
2016-01-14 18:31:45,951 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] CFB11C367CDB4D6C992AEDD08213B145
2016-01-14 18:31:45,951 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@7a3c1298; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@7de8f7d9; comparator=cascading.tuple.hadoop.util.TupleComparator@49e80823; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:45,951 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:45,951 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:45,951 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:45,974 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:45,974 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:45,974 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:45,975 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:45,975 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:45,977 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:45,977 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:45,977 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: AF17A5BE952547B0A81574220178C57D, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:45,980 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796305751_0001_1_00_000000_0
2016-01-14 18:31:45,981 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: AF17A5BE952547B0A81574220178C57D, mem on close (mb), free: 283, total: 561, max: 672
2016-01-14 18:31:45,981 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:45,981 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:45,981 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:45,981 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:45,983 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:45 Completed running task attempt: attempt_1452796305751_0001_1_00_000000_0
2016-01-14 18:31:45,985 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796305751_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:45,986 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796305977, Process_Duration=3, Process_End_Time=1452796305980, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:45,986 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796305751_0001_1_00, eventCount=1
2016-01-14 18:31:45,986 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:45,986 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,986 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=AF17A5BE952547B0A81574220178C57D, taskAttemptId=attempt_1452796305751_0001_1_00_000000_0, startTime=1452796305935, finishTime=1452796305986, timeTaken=51, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305977, Process_Duration=3, Process_End_Time=1452796305980, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:45,986 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305751_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:45,986 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:45,987 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796305751_0001_1_00_000000_0
2016-01-14 18:31:45,987 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:45,987 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:TASK_FINISHED]: vertexName=AF17A5BE952547B0A81574220178C57D, taskId=task_1452796305751_0001_1_00_000000, startTime=1452796305935, finishTime=1452796305987, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1452796305751_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305977, Process_Duration=3, Process_End_Time=1452796305980, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:45,987 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796305751_0001_00_000001
2016-01-14 18:31:45,987 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305751_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:45,987 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D] : 1
2016-01-14 18:31:45,987 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D]
2016-01-14 18:31:45,987 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:45,988 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:45,988 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:VERTEX_FINISHED]: vertexName=AF17A5BE952547B0A81574220178C57D, vertexId=vertex_1452796305751_0001_1_00, initRequestedTime=1452796305912, initedTime=1452796305924, startRequestedTime=1452796305925, startedTime=1452796305925, finishTime=1452796305987, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305977, Process_Duration=3, Process_End_Time=1452796305980, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796305935, firstTasksToStart=[ task_1452796305751_0001_1_00_000000 ], lastTaskFinishTime=1452796305986, lastTasksToFinish=[ task_1452796305751_0001_1_00_000000 ], minTaskDuration=51, maxTaskDuration=51, avgTaskDuration=51.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796305751_0001_1_00_000000 ], longestDurationTasks=[ task_1452796305751_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:45,988 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:45,988 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] attempt: attempt_1452796305751_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:45,988 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: BEFF6B05109E4AA594A5BBF15B6C939D
2016-01-14 18:31:45,988 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D]
2016-01-14 18:31:45,989 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:45,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796305751_0001_00_000001, stoppedTime=1452796305989, exitStatus=0
2016-01-14 18:31:45,989 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:45,989 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:45,989 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:45,989 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:TASK_STARTED]: vertexName=BEFF6B05109E4AA594A5BBF15B6C939D, taskId=task_1452796305751_0001_1_01_000000, scheduledTime=1452796305989, launchTime=1452796305989
2016-01-14 18:31:45,989 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:45,989 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305751_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:45,990 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796305751_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:45,990 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D]
2016-01-14 18:31:45,990 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:45,990 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: BEFF6B05109E4AA594A5BBF15B6C939D, VertexParallelism: 1, TaskAttemptID:attempt_1452796305751_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AF17A5BE952547B0A81574220178C57D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=02C6652ECA764D2893967E498B66FE4F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:45,991 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305751_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:45,991 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:45,991 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796305751_0001_00_000002
2016-01-14 18:31:45,992 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:45,993 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796305751_0001_00_000002
2016-01-14 18:31:45,993 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:45,994 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:45,995 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:45,995 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796305751_0001_00_000002, launchTime=1452796305994
2016-01-14 18:31:45,995 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796305751_0001_00_000002
2016-01-14 18:31:45,996 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796305751_0001_1_01_000000_0] to container: [container_1452796305751_0001_00_000002]
2016-01-14 18:31:45,996 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:45,996 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796305751_0001_00_000002 given task: attempt_1452796305751_0001_1_01_000000_0
2016-01-14 18:31:45,997 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796305751_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796305751_0001_1_01_000000_0
2016-01-14 18:31:45,997 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796305751_0001_1_01_000000_0] started. Is using containerId: [container_1452796305751_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:45,997 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:45,997 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=BEFF6B05109E4AA594A5BBF15B6C939D, taskAttemptId=attempt_1452796305751_0001_1_01_000000_0, startTime=1452796305997, containerId=container_1452796305751_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796305751_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:45,997 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305751_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:45 Starting to run new task attempt: attempt_1452796305751_0001_1_01_000000_0
2016-01-14 18:31:45,998 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305751_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:45,998 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796305751_0001_1_01_000000_0
2016-01-14 18:31:45,998 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:45,998 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:45,998 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:45,999 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:45,999 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:45,999 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796305751_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:45,999 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796305751_0001_00_000001: 12 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:45,999 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796305751_0001_00_000001, Exiting
2016-01-14 18:31:45,999 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796305751_0001_00_000001
2016-01-14 18:31:45,999 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796305751_0001_00_000001
2016-01-14 18:31:45,999 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:46,000 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5] every/(1/1) ...yplatform/ALL_VALUES_SWAP, VertexName: BEFF6B05109E4AA594A5BBF15B6C939D, VertexParallelism: 1, TaskAttemptID:attempt_1452796305751_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=AF17A5BE952547B0A81574220178C57D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=02C6652ECA764D2893967E498B66FE4F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:46,000 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:46,001 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796305751_0001_1_01_000000_0
2016-01-14 18:31:46,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:46,001 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,001 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796305751_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:46,001 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796305751_0001_00_000001 completed successfully
2016-01-14 18:31:46,001 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:46,002 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796305751_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:46,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=AF17A5BE952547B0A81574220178C57D, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:46,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:46,004 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: AF17A5BE952547B0A81574220178C57D
2016-01-14 18:31:46,003 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=02C6652ECA764D2893967E498B66FE4F, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:46,002 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,006 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:46,006 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:46,006 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: AF17A5BE952547B0A81574220178C57D
2016-01-14 18:31:46,006 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: AF17A5BE952547B0A81574220178C57D
2016-01-14 18:31:46,008 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:46,009 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:46,010 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 02C6652ECA764D2893967E498B66FE4F
2016-01-14 18:31:46,010 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:46,011 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 02C6652ECA764D2893967E498B66FE4F
2016-01-14 18:31:46,012 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: BEFF6B05109E4AA594A5BBF15B6C939D, ordinal: 1
2016-01-14 18:31:46,012 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEFF6B05109E4AA594A5BBF15B6C939D, mem on start (mb), free: 277, total: 561, max: 672
2016-01-14 18:31:46,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:46,012 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:46,019 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:46,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 02C6652ECA764D2893967E498B66FE4F
2016-01-14 18:31:46,019 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:46,019 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:46,019 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:46,019 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:46,019 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:46,019 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:46,019 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:46,020 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, AF17A5BE952547B0A81574220178C57D, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:46,020 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 02C6652ECA764D2893967E498B66FE4F, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:46,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:46,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: AF17A5BE952547B0A81574220178C57D being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:46,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:46,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:46,020 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: AF17A5BE952547B0A81574220178C57D
2016-01-14 18:31:46,021 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:46,021 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:46,022 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: AF17A5BE952547B0A81574220178C57D with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:46,022 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:46,022 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:46,022 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:46,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: AF17A5BE952547B0A81574220178C57D
2016-01-14 18:31:46,023 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:46,023 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796305751_0001_1_01_000000_0
2016-01-14 18:31:46,024 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796305751_0001_1_00_000000_0_10315, runDuration: 45573, ]
2016-01-14 18:31:46,026 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796305751_0001_1_00_000000_0_10315] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796306026, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:46,026 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : AF17A5BE952547B0A81574220178C57D
2016-01-14 18:31:46,026 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:46,026 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:46,026 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [AF17A5BE952547B0A81574220178C57D] #53 in 1ms
2016-01-14 18:31:46,027 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: CFB11C367CDB4D6C992AEDD08213B145
2016-01-14 18:31:46,027 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [AF17A5BE952547B0A81574220178C57D] #53, status:false, isInterrupted:false
2016-01-14 18:31:46,027 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP"], id: 02C6652ECA764D2893967E498B66FE4F
2016-01-14 18:31:46,027 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:46,028 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:46,028 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:46,028 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:46,028 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:46,028 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : AF17A5BE952547B0A81574220178C57D
2016-01-14 18:31:46,028 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:46,029 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_VALUES_SWAP"]
2016-01-14 18:31:46,029 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] CFB11C367CDB4D6C992AEDD08213B145, for 1 inputs
2016-01-14 18:31:46,029 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: BEFF6B05109E4AA594A5BBF15B6C939D, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:46,030 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@a8457f6; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@76aadcde
2016-01-14 18:31:46,030 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:46,032 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:46,033 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:46,035 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:46,035 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:46,035 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:46,035 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796305751_0001_1_01_000000_0
2016-01-14 18:31:46,035 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796305751_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:46,036 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796305751_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:46,038 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:46,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=BEFF6B05109E4AA594A5BBF15B6C939D, taskAttemptId=attempt_1452796305751_0001_1_01_000000_0, startTime=1452796305997, finishTime=1452796306038, timeTaken=41, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796306029, Process_Duration=7, Process_End_Time=1452796306036, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:46,039 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305751_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:46,039 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796305751_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:46,039 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:TASK_FINISHED]: vertexName=BEFF6B05109E4AA594A5BBF15B6C939D, taskId=task_1452796305751_0001_1_01_000000, startTime=1452796305997, finishTime=1452796306039, timeTaken=42, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:46,039 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796305751_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:46,039 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] : 1
2016-01-14 18:31:46,039 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] because task failed: task_1452796305751_0001_1_01_000000
2016-01-14 18:31:46,039 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:46,039 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:46,040 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:46,040 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:46,040 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D]
2016-01-14 18:31:46,040 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=02C6652ECA764D2893967E498B66FE4F, vertexId=vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D]
2016-01-14 18:31:46,042 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:46,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:VERTEX_FINISHED]: vertexName=BEFF6B05109E4AA594A5BBF15B6C939D, vertexId=vertex_1452796305751_0001_1_01, initRequestedTime=1452796305914, initedTime=1452796305920, startRequestedTime=1452796305927, startedTime=1452796305927, finishTime=1452796306042, timeTaken=115, status=FAILED, diagnostics=Task failed, taskId=task_1452796305751_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796305997, firstTasksToStart=[ task_1452796305751_0001_1_01_000000 ], lastTaskFinishTime=1452796306038, lastTasksToFinish=[ task_1452796305751_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:46,042 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:46,042 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796305751_0001_00_000002, stoppedTime=1452796306042, exitStatus=0
2016-01-14 18:31:46,042 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:46,043 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796305751_0001_1_01_000000_0 failed: true container: container_1452796305751_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:46,043 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:46,043 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:46,043 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796305751_0001_1
2016-01-14 18:31:46,043 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796305751_0001_1_00 [AF17A5BE952547B0A81574220178C57D]
2016-01-14 18:31:46,043 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 02C6652ECA764D2893967E498B66FE4F for vertex: vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D]
2016-01-14 18:31:46,044 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796305751_0001_1, queueSize=0
2016-01-14 18:31:46,045 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796305751_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796305751_0001_1, startTime=1452796305912, finishTime=1452796306043, timeTaken=131, status=FAILED, diagnostics=Vertex failed, vertexName=BEFF6B05109E4AA594A5BBF15B6C939D, vertexId=vertex_1452796305751_0001_1_01, diagnostics=[Task failed, taskId=task_1452796305751_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: [NONE], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796305751_0001_1_01 [BEFF6B05109E4AA594A5BBF15B6C939D] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=853, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796305977, Process_Duration=3, Process_End_Time=1452796305980, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=1, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:46,045 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796305751_0001_1 finished with state: FAILED
2016-01-14 18:31:46,045 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796305751_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:46,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:46,045 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:46,046 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796305751_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:46 Completed Dag: dag_1452796305751_0001_1
2016-01-14 18:31:46,046 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796305751_0001_1, dagState=FAILED
2016-01-14 18:31:46,046 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796305751_0001_1_post
2016-01-14 18:31:46,046 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:46,046 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:46,046 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:46,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=588775424, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=4, LAST_EVENT_RECEIVED=4][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796306029, Process_Duration=7, Process_End_Time=1452796306036, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:46,047 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: BEFF6B05109E4AA594A5BBF15B6C939D, mem on close (mb), free: 272, total: 561, max: 672
2016-01-14 18:31:46,047 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=BEFF6B05109E4AA594A5BBF15B6C939D, index=1
2016-01-14 18:31:46,047 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: AF17A5BE952547B0A81574220178C57D
2016-01-14 18:31:46,047 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:46,048 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:46,048 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:46,048 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:46,048 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:46,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=BEFF6B05109E4AA594A5BBF15B6C939D, sourceVertex=AF17A5BE952547B0A81574220178C57D
2016-01-14 18:31:46,048 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=BEFF6B05109E4AA594A5BBF15B6C939D, sourceVertex=02C6652ECA764D2893967E498B66FE4F
2016-01-14 18:31:46,049 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796305751_0001_00_000002
2016-01-14 18:31:46,049 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796305751_0001_00_000002
2016-01-14 18:31:46,049 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796305751_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:46,049 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796305751_0001_00_000002 completed but with errors
2016-01-14 18:31:46,049 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796305751_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:46,049 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796305751_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:46,049 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796305751_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:46,050 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...yplatform/ALL_VALUES_SWAP, applicationId=application_1452796305751_0001
2016-01-14 18:31:46,051 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:46,052 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:46,052 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:46,052 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:46,052 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:46,052 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:46,052 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:46,053 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:46,053 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:46,053 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:46,057 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:46,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:46,057 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:46,061 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/FF6A1B60CC46443195CC56365E2153B6/F2A78A5D39B94BF39F549890C8B945E5/.tez/application_1452796305751_0001
2016-01-14 18:31:46,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:46,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...yplatform/ALL_VALUES_SWAP
2016-01-14 18:31:46,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:46,062 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.332
2016-01-14 18:31:46,063 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_VALUES_SWAP
2016-01-14 18:31:46,087 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:46,087 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:46,115 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.005
2016-01-14 18:31:46,124 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.014
2016-01-14 18:31:46,124 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:46,124 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:46,125 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:46,139 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:46,143 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1
2016-01-14 18:31:46,184 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:46,184 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:46,185 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1"]
2016-01-14 18:31:46,185 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:46,185 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:46,185 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:46,185 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 18:31:46,186 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:46,198 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:46,202 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:46,202 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:46,202 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:46,211 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123/.tez/application_1452796306202_0001 doesn't exist and is created
2016-01-14 18:31:46,225 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:46,225 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:46,225 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123/.tez/application_1452796306202_0001
2016-01-14 18:31:46,227 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796306202_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:46,238 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:46,239 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:46,239 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:46,239 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:46,240 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123/.tez/application_1452796306202_0001/localmode-log-dir
2016-01-14 18:31:46,240 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123/.tez/application_1452796306202_0001/localmode-log-dir/history.txt.appattempt_1452796306202_0001_000000, maxErrors=10
2016-01-14 18:31:46,240 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:46,240 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796306202_0001_000000, appSubmitTime=1452796306226, launchTime=1452796306226
2016-01-14 18:31:46,244 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:46,244 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:46,245 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:46,245 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:46,246 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:38829
2016-01-14 18:31:46,247 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:46,249 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796306202_0001_000000, startTime=1452796306249
2016-01-14 18:31:46,249 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:46,249 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:46,325 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:46,326 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:46,326 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1452796306202_0001, dagName=[8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 18:31:46,340 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 18:31:46,342 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796306202_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123/.tez/application_1452796306202_0001/localmode-log-dir/dag_1452796306202_0001_1.dot
2016-01-14 18:31:46,342 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123/.tez/application_1452796306202_0001/localmode-log-dir/dag_1452796306202_0001_1-tez-dag.pb.txt
2016-01-14 18:31:46,349 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796306202_0001_1
2016-01-14 18:31:46,349 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:46,350 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 18:31:46 Running Dag: dag_1452796306202_0001_1
2016-01-14 18:31:46,359 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796306202_0001_1, submitTime=1452796306341
2016-01-14 18:31:46,359 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:46,359 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex E25F973D048847508BE13A2309A5BC88
2016-01-14 18:31:46,360 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:46,360 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796306202_0001_1, initTime=1452796306359
2016-01-14 18:31:46,360 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:46,360 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796306202_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:46,361 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:DAG_STARTED]: dagID=dag_1452796306202_0001_1, startTime=1452796306360
2016-01-14 18:31:46,361 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:46,361 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1452796306202_0001, dagName=[8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123] every/(1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 18:31:46,361 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:46,361 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796306202_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:46,361 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: B6308F6CCE1545DA8FCB9A8C6D2C3B7B : {79EDC0F35B434E39B6F819A15F433775={InputName=79EDC0F35B434E39B6F819A15F433775}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:46,361 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 79EDC0F35B434E39B6F819A15F433775, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:46,361 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B]
2016-01-14 18:31:46,361 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B]
2016-01-14 18:31:46,361 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B]
2016-01-14 18:31:46,362 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B]. Starting root input initializers: 1
2016-01-14 18:31:46,363 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:46,363 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88]
2016-01-14 18:31:46,364 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 79EDC0F35B434E39B6F819A15F433775 on vertex vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B]
2016-01-14 18:31:46,366 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:46,366 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:46,366 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88]
2016-01-14 18:31:46,366 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88]
2016-01-14 18:31:46,366 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88]
2016-01-14 18:31:46,367 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=C277E22431B6460599349CA4F0A4D154, vertexId=vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:46,367 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=C277E22431B6460599349CA4F0A4D154, vertexId=vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88]
2016-01-14 18:31:46,367 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 79EDC0F35B434E39B6F819A15F433775 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:46,368 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:46,368 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:46,369 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=C277E22431B6460599349CA4F0A4D154, vertexName=E25F973D048847508BE13A2309A5BC88, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:46,369 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=C277E22431B6460599349CA4F0A4D154, vertexId=vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88]
2016-01-14 18:31:46,370 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:46,370 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 79EDC0F35B434E39B6F819A15F433775 on vertex vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B]
2016-01-14 18:31:46,370 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:VERTEX_INITIALIZED]: vertexName=E25F973D048847508BE13A2309A5BC88, vertexId=vertex_1452796306202_0001_1_01, initRequestedTime=1452796306363, initedTime=1452796306370, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:46,370 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:46,371 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:46,371 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:46,371 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {79EDC0F35B434E39B6F819A15F433775=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:46,371 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] parallelism set to 1
2016-01-14 18:31:46,372 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:46,372 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796306202_0001_1_00, eventCount=1
2016-01-14 18:31:46,372 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:46,372 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B6308F6CCE1545DA8FCB9A8C6D2C3B7B, vertexId=vertex_1452796306202_0001_1_00, initRequestedTime=1452796306361, initedTime=1452796306372, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:46,373 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] with distanceFromRoot: 0
2016-01-14 18:31:46,373 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:46,373 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in B6308F6CCE1545DA8FCB9A8C6D2C3B7B
2016-01-14 18:31:46,373 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B]
2016-01-14 18:31:46,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:46,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796306202_0001_1_00, startRequestedTime=1452796306373, startedTime=1452796306373
2016-01-14 18:31:46,374 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:46,374 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:46,374 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:TASK_STARTED]: vertexName=B6308F6CCE1545DA8FCB9A8C6D2C3B7B, taskId=task_1452796306202_0001_1_00_000000, scheduledTime=1452796306374, launchTime=1452796306374
2016-01-14 18:31:46,375 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306202_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:46,375 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796306202_0001_1_00 for vertex: vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] numStartedSources: 1 numSources: 1
2016-01-14 18:31:46,375 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] with distanceFromRoot: 1
2016-01-14 18:31:46,375 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B]
2016-01-14 18:31:46,376 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: B6308F6CCE1545DA8FCB9A8C6D2C3B7B in vertex: E25F973D048847508BE13A2309A5BC88
2016-01-14 18:31:46,376 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: E25F973D048847508BE13A2309A5BC88 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:46,376 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:46,376 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796306202_0001_1_01, startRequestedTime=1452796306375, startedTime=1452796306375
2016-01-14 18:31:46,376 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:46,377 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: B6308F6CCE1545DA8FCB9A8C6D2C3B7B, VertexParallelism: 1, TaskAttemptID:attempt_1452796306202_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=79EDC0F35B434E39B6F819A15F433775, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E25F973D048847508BE13A2309A5BC88, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:46,377 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306202_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:46,377 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:46,378 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:46,378 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:46,378 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:46,378 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796306202_0001_00_000001
2016-01-14 18:31:46,378 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:46,380 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796306202_0001_00_000001
2016-01-14 18:31:46,380 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:46,381 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:46,381 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:46,381 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796306202_0001, with dag id: dag_1452796306202_0001_1
2016-01-14 18:31:46,382 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796306202_0001_00_000001, launchTime=1452796306381
2016-01-14 18:31:46,383 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796306202_0001_00_000001
2016-01-14 18:31:46,383 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796306202_0001_1_00_000000_0] to container: [container_1452796306202_0001_00_000001]
2016-01-14 18:31:46,383 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:46,384 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796306202_0001_1_00_000000_0] started. Is using containerId: [container_1452796306202_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:46,384 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B6308F6CCE1545DA8FCB9A8C6D2C3B7B, taskAttemptId=attempt_1452796306202_0001_1_00_000000_0, startTime=1452796306384, containerId=container_1452796306202_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796306202_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:46,384 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:46,384 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306202_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:46,384 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306202_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:46,384 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796306202_0001_00_000001 given task: attempt_1452796306202_0001_1_00_000000_0
2016-01-14 18:31:46,385 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796306202_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796306202_0001_1_00_000000_0
2016-01-14 18:31:46 Starting to run new task attempt: attempt_1452796306202_0001_1_00_000000_0
2016-01-14 18:31:46,388 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796306202_0001_1_00_000000_0
2016-01-14 18:31:46,388 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:46,388 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:46,389 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:46,389 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:46,389 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:46,389 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:46,389 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: B6308F6CCE1545DA8FCB9A8C6D2C3B7B, VertexParallelism: 1, TaskAttemptID:attempt_1452796306202_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=79EDC0F35B434E39B6F819A15F433775, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=E25F973D048847508BE13A2309A5BC88, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:46,390 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:46,390 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796306202_0001_1_00_000000_0
2016-01-14 18:31:46,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:46,391 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796306202_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:46,391 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=79EDC0F35B434E39B6F819A15F433775, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:46,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=E25F973D048847508BE13A2309A5BC88, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:46,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:46,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: E25F973D048847508BE13A2309A5BC88
2016-01-14 18:31:46,392 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:46,393 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 79EDC0F35B434E39B6F819A15F433775
2016-01-14 18:31:46,393 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 79EDC0F35B434E39B6F819A15F433775
2016-01-14 18:31:46,395 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:46,395 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 79EDC0F35B434E39B6F819A15F433775
2016-01-14 18:31:46,395 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 79EDC0F35B434E39B6F819A15F433775
2016-01-14 18:31:46,396 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:46,396 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:46,396 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: E25F973D048847508BE13A2309A5BC88
2016-01-14 18:31:46,396 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: E25F973D048847508BE13A2309A5BC88
2016-01-14 18:31:46,399 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B6308F6CCE1545DA8FCB9A8C6D2C3B7B, ordinal: 0
2016-01-14 18:31:46,399 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B6308F6CCE1545DA8FCB9A8C6D2C3B7B, mem on start (mb), free: 186, total: 561, max: 672
2016-01-14 18:31:46,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:46,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:46,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:46,399 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:46,400 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:46,400 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:46,400 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:46,400 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:46,400 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:46,401 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 79EDC0F35B434E39B6F819A15F433775, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:46,401 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, E25F973D048847508BE13A2309A5BC88, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:46,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:46,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 79EDC0F35B434E39B6F819A15F433775 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:46,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:46,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:46,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 79EDC0F35B434E39B6F819A15F433775
2016-01-14 18:31:46,402 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:46,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 79EDC0F35B434E39B6F819A15F433775
2016-01-14 18:31:46,402 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:46,403 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796306202_0001_1_00_000000_0
2016-01-14 18:31:46,403 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:46,404 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@2263dbc1
2016-01-14 18:31:46,404 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@2263dbc1
2016-01-14 18:31:46,405 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:46,405 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:46,406 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:46,407 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 79EDC0F35B434E39B6F819A15F433775
2016-01-14 18:31:46,407 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 74E070550CF7497A9D042F8F782DC417
2016-01-14 18:31:46,407 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 74E070550CF7497A9D042F8F782DC417
2016-01-14 18:31:46,407 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@22cb737a; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5b0a606e; comparator=cascading.tuple.hadoop.util.TupleComparator@6410474; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:46,408 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:46,408 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:46,408 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:46,429 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:46,430 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:46,431 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:46,431 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:46,431 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:46,432 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:46,432 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:46,433 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B6308F6CCE1545DA8FCB9A8C6D2C3B7B, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:46,435 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796306202_0001_1_00_000000_0
2016-01-14 18:31:46,435 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B6308F6CCE1545DA8FCB9A8C6D2C3B7B, mem on close (mb), free: 283, total: 564, max: 672
2016-01-14 18:31:46,435 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:46,435 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:46,436 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:46,436 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:46,437 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:46 Completed running task attempt: attempt_1452796306202_0001_1_00_000000_0
2016-01-14 18:31:46,439 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796306202_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:46,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796306202_0001_1_00, eventCount=1
2016-01-14 18:31:46,440 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B6308F6CCE1545DA8FCB9A8C6D2C3B7B, taskAttemptId=attempt_1452796306202_0001_1_00_000000_0, startTime=1452796306384, finishTime=1452796306440, timeTaken=56, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796306433, Process_Duration=2, Process_End_Time=1452796306435, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:46,440 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306202_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:46,440 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796306202_0001_1_00_000000_0
2016-01-14 18:31:46,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:TASK_FINISHED]: vertexName=B6308F6CCE1545DA8FCB9A8C6D2C3B7B, taskId=task_1452796306202_0001_1_00_000000, startTime=1452796306384, finishTime=1452796306440, timeTaken=56, status=SUCCEEDED, successfulAttemptID=attempt_1452796306202_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796306433, Process_Duration=2, Process_End_Time=1452796306435, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:46,441 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306202_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:46,441 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:46,441 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] : 1
2016-01-14 18:31:46,441 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B]
2016-01-14 18:31:46,441 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:46,441 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:46,441 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:VERTEX_FINISHED]: vertexName=B6308F6CCE1545DA8FCB9A8C6D2C3B7B, vertexId=vertex_1452796306202_0001_1_00, initRequestedTime=1452796306361, initedTime=1452796306372, startRequestedTime=1452796306373, startedTime=1452796306373, finishTime=1452796306441, timeTaken=68, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796306433, Process_Duration=2, Process_End_Time=1452796306435, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796306384, firstTasksToStart=[ task_1452796306202_0001_1_00_000000 ], lastTaskFinishTime=1452796306440, lastTasksToFinish=[ task_1452796306202_0001_1_00_000000 ], minTaskDuration=56, maxTaskDuration=56, avgTaskDuration=56.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796306202_0001_1_00_000000 ], longestDurationTasks=[ task_1452796306202_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:46,441 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:46,442 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:46,442 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] attempt: attempt_1452796306202_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:46,442 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: E25F973D048847508BE13A2309A5BC88
2016-01-14 18:31:46,442 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88]
2016-01-14 18:31:46,442 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:46,442 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 28 [[org.apache.tez.common.counters.DAGCounter RACK_LOCAL_TASKS=1][File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796306433, Process_Duration=2, Process_End_Time=1452796306435, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:46,442 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796306202_0001_00_000001, stoppedTime=1452796306442, exitStatus=0
2016-01-14 18:31:46,442 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:46,442 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:46,442 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:46,443 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:TASK_STARTED]: vertexName=E25F973D048847508BE13A2309A5BC88, taskId=task_1452796306202_0001_1_01_000000, scheduledTime=1452796306443, launchTime=1452796306443
2016-01-14 18:31:46,443 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:46,443 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:46,443 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:46,443 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306202_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:46,443 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796306202_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:46,444 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88]
2016-01-14 18:31:46,444 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:46,444 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: E25F973D048847508BE13A2309A5BC88, VertexParallelism: 1, TaskAttemptID:attempt_1452796306202_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B6308F6CCE1545DA8FCB9A8C6D2C3B7B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C277E22431B6460599349CA4F0A4D154, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:46,444 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306202_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:46,445 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:46,443 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796306202_0001_00_000001
2016-01-14 18:31:46,445 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796306202_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:46,445 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796306202_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:46,445 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796306202_0001_00_000001, Exiting
2016-01-14 18:31:46,446 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796306202_0001_00_000001
2016-01-14 18:31:46,446 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796306202_0001_00_000001
2016-01-14 18:31:46,446 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796306202_0001_00_000002
2016-01-14 18:31:46,446 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:46,446 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796306202_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:46,447 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796306202_0001_00_000001 completed successfully
2016-01-14 18:31:46,447 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796306202_0001_00_000002
2016-01-14 18:31:46,448 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:46,448 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-teztask.properties,hadoop-metrics2.properties
2016-01-14 18:31:46,450 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:46,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:46,450 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796306202_0001_00_000002, launchTime=1452796306450
2016-01-14 18:31:46,451 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796306202_0001_00_000002
2016-01-14 18:31:46,453 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796306202_0001_1_01_000000_0] to container: [container_1452796306202_0001_00_000002]
2016-01-14 18:31:46,453 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:46,453 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796306202_0001_1_01_000000_0] started. Is using containerId: [container_1452796306202_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:46,453 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=E25F973D048847508BE13A2309A5BC88, taskAttemptId=attempt_1452796306202_0001_1_01_000000_0, startTime=1452796306453, containerId=container_1452796306202_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796306202_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:46,454 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306202_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:46,454 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306202_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:46,454 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796306202_0001_00_000002 given task: attempt_1452796306202_0001_1_01_000000_0
2016-01-14 18:31:46,454 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796306202_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796306202_0001_1_01_000000_0
2016-01-14 18:31:46,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:46 Starting to run new task attempt: attempt_1452796306202_0001_1_01_000000_0
2016-01-14 18:31:46,455 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796306202_0001_1_01_000000_0
2016-01-14 18:31:46,455 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:46,455 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:46,455 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:46,455 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:46,455 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:46,456 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:46,456 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123] every/(1/1) ...emblyplatform/ALL_ARGS_-1, VertexName: E25F973D048847508BE13A2309A5BC88, VertexParallelism: 1, TaskAttemptID:attempt_1452796306202_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=B6308F6CCE1545DA8FCB9A8C6D2C3B7B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=C277E22431B6460599349CA4F0A4D154, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:46,456 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:46,458 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796306202_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:46,459 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796306202_0001_1_01_000000_0
2016-01-14 18:31:46,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:46,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=C277E22431B6460599349CA4F0A4D154, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:46,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=B6308F6CCE1545DA8FCB9A8C6D2C3B7B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:46,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:46,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: C277E22431B6460599349CA4F0A4D154
2016-01-14 18:31:46,461 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:46,461 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: C277E22431B6460599349CA4F0A4D154
2016-01-14 18:31:46,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:46,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: B6308F6CCE1545DA8FCB9A8C6D2C3B7B
2016-01-14 18:31:46,463 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:46,463 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:46,463 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: B6308F6CCE1545DA8FCB9A8C6D2C3B7B
2016-01-14 18:31:46,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: B6308F6CCE1545DA8FCB9A8C6D2C3B7B
2016-01-14 18:31:46,465 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:46,469 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:46,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: C277E22431B6460599349CA4F0A4D154
2016-01-14 18:31:46,470 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: E25F973D048847508BE13A2309A5BC88, ordinal: 1
2016-01-14 18:31:46,470 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E25F973D048847508BE13A2309A5BC88, mem on start (mb), free: 277, total: 564, max: 672
2016-01-14 18:31:46,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:46,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:46,470 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:46,470 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:46,470 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:46,470 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:46,470 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:46,471 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:46,471 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:46,471 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, C277E22431B6460599349CA4F0A4D154, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:46,471 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, B6308F6CCE1545DA8FCB9A8C6D2C3B7B, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:46,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:46,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: B6308F6CCE1545DA8FCB9A8C6D2C3B7B
2016-01-14 18:31:46,471 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:46,471 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:46,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: B6308F6CCE1545DA8FCB9A8C6D2C3B7B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:46,471 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: B6308F6CCE1545DA8FCB9A8C6D2C3B7B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:46,472 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:46,472 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:46,472 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:46,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:46,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:46,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: B6308F6CCE1545DA8FCB9A8C6D2C3B7B
2016-01-14 18:31:46,473 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:46,473 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796306202_0001_1_01_000000_0
2016-01-14 18:31:46,474 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796306202_0001_1_00_000000_0_10320, runDuration: 45971, ]
2016-01-14 18:31:46,475 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796306202_0001_1_00_000000_0_10320] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796306475, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:46,475 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : B6308F6CCE1545DA8FCB9A8C6D2C3B7B
2016-01-14 18:31:46,475 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:46,475 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:46,476 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] #54 in 2ms
2016-01-14 18:31:46,476 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [B6308F6CCE1545DA8FCB9A8C6D2C3B7B] #54, status:false, isInterrupted:false
2016-01-14 18:31:46,476 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 74E070550CF7497A9D042F8F782DC417
2016-01-14 18:31:46,476 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1"], id: C277E22431B6460599349CA4F0A4D154
2016-01-14 18:31:46,476 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:46,476 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_-1"]
2016-01-14 18:31:46,476 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:46,476 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:46,476 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:46,476 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 74E070550CF7497A9D042F8F782DC417, for 1 inputs
2016-01-14 18:31:46,477 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:46,477 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : B6308F6CCE1545DA8FCB9A8C6D2C3B7B
2016-01-14 18:31:46,477 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:46,477 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: E25F973D048847508BE13A2309A5BC88, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:46,477 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@42fcac6; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@5da98f4f
2016-01-14 18:31:46,478 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:46,478 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:46,478 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:46,478 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:46,479 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796306202_0001_1_01_000000_0
2016-01-14 18:31:46,479 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796306202_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:46,479 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796306202_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:46,481 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:46,481 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=E25F973D048847508BE13A2309A5BC88, taskAttemptId=attempt_1452796306202_0001_1_01_000000_0, startTime=1452796306453, finishTime=1452796306481, timeTaken=28, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796306477, Process_Duration=2, Process_End_Time=1452796306479, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:46,481 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306202_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:46,481 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796306202_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:46,481 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:TASK_FINISHED]: vertexName=E25F973D048847508BE13A2309A5BC88, taskId=task_1452796306202_0001_1_01_000000, startTime=1452796306453, finishTime=1452796306481, timeTaken=28, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:46,481 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306202_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:46,482 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] : 1
2016-01-14 18:31:46,482 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] because task failed: task_1452796306202_0001_1_01_000000
2016-01-14 18:31:46,482 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:46,482 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:46,482 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88]
2016-01-14 18:31:46,482 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=C277E22431B6460599349CA4F0A4D154, vertexId=vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88]
2016-01-14 18:31:46,483 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:VERTEX_FINISHED]: vertexName=E25F973D048847508BE13A2309A5BC88, vertexId=vertex_1452796306202_0001_1_01, initRequestedTime=1452796306363, initedTime=1452796306370, startRequestedTime=1452796306375, startedTime=1452796306375, finishTime=1452796306483, timeTaken=108, status=FAILED, diagnostics=Task failed, taskId=task_1452796306202_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796306453, firstTasksToStart=[ task_1452796306202_0001_1_01_000000 ], lastTaskFinishTime=1452796306481, lastTasksToFinish=[ task_1452796306202_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:46,483 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:46,484 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:46,484 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:46,484 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796306202_0001_1
2016-01-14 18:31:46,484 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: C277E22431B6460599349CA4F0A4D154 for vertex: vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88]
2016-01-14 18:31:46,484 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:46,484 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796306202_0001_1_00 [B6308F6CCE1545DA8FCB9A8C6D2C3B7B]
2016-01-14 18:31:46,485 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796306202_0001_1, queueSize=0
2016-01-14 18:31:46,485 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796306202_0001_1, startTime=1452796306360, finishTime=1452796306484, timeTaken=124, status=FAILED, diagnostics=Vertex failed, vertexName=E25F973D048847508BE13A2309A5BC88, vertexId=vertex_1452796306202_0001_1_01, diagnostics=[Task failed, taskId=task_1452796306202_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796306202_0001_1_01 [E25F973D048847508BE13A2309A5BC88] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796306433, Process_Duration=2, Process_End_Time=1452796306435, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:46,486 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796306202_0001_1 finished with state: FAILED
2016-01-14 18:31:46,486 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796306202_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:46,486 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306202_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796306202_0001_00_000002, stoppedTime=1452796306486, exitStatus=0
2016-01-14 18:31:46,486 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:46,486 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796306202_0001_1_01_000000_0 failed: true container: container_1452796306202_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:46 Completed Dag: dag_1452796306202_0001_1
2016-01-14 18:31:46,486 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796306202_0001_1, dagState=FAILED
2016-01-14 18:31:46,487 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796306202_0001_1_post
2016-01-14 18:31:46,487 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:46,487 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:46,487 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796306202_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:46,487 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:46,481 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796306477, Process_Duration=2, Process_End_Time=1452796306479, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:46,481 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:46,488 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:46,488 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:46,489 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:46,489 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: E25F973D048847508BE13A2309A5BC88, mem on close (mb), free: 273, total: 564, max: 672
2016-01-14 18:31:46,489 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=E25F973D048847508BE13A2309A5BC88, index=1
2016-01-14 18:31:46,490 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: B6308F6CCE1545DA8FCB9A8C6D2C3B7B
2016-01-14 18:31:46,490 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:46,490 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:46,490 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:46,490 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:46,491 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:46,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=E25F973D048847508BE13A2309A5BC88, sourceVertex=B6308F6CCE1545DA8FCB9A8C6D2C3B7B
2016-01-14 18:31:46,491 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=E25F973D048847508BE13A2309A5BC88, sourceVertex=C277E22431B6460599349CA4F0A4D154
2016-01-14 18:31:46,491 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796306202_0001_00_000002
2016-01-14 18:31:46,491 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796306202_0001_00_000002
2016-01-14 18:31:46,492 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796306202_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:46,492 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796306202_0001_00_000002 completed but with errors
2016-01-14 18:31:46,492 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796306202_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:46,492 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306202_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:46,492 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306202_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:46,500 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...emblyplatform/ALL_ARGS_-1, applicationId=application_1452796306202_0001
2016-01-14 18:31:46,501 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:46,501 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:46,501 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:46,501 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:46,501 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:46,503 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:46,503 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:46,503 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:46,503 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:46,503 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:46,504 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:46,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:46,504 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:46,507 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/8004922782B84B21ADE453BE225E26AD/A801596BD8AD47DDA6B4A0491E4C6123/.tez/application_1452796306202_0001
2016-01-14 18:31:46,507 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:46,507 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...emblyplatform/ALL_ARGS_-1
2016-01-14 18:31:46,508 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:46,508 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.324
2016-01-14 18:31:46,508 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_-1
2016-01-14 18:31:46,532 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:46,532 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:46,549 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:46,549 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:46,558 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.002
2016-01-14 18:31:46,559 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.003
2016-01-14 18:31:46,560 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:46,560 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:46,560 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_'field'
2016-01-14 18:31:46,580 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:46,581 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:46,609 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.006
2016-01-14 18:31:46,614 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.011
2016-01-14 18:31:46,615 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:46,615 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:46,615 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:46,628 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:46,632 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS
2016-01-14 18:31:46,672 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:46,673 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:46,673 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS"]
2016-01-14 18:31:46,673 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:46,674 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:46,674 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:46,674 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 18:31:46,675 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:46,686 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:46,690 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:46,690 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:46,691 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:46,699 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F/.tez/application_1452796306690_0001 doesn't exist and is created
2016-01-14 18:31:46,712 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:46,712 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:46,713 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F/.tez/application_1452796306690_0001
2016-01-14 18:31:46,714 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796306690_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:46,724 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:46,725 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:46,726 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:46,726 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:46,726 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F/.tez/application_1452796306690_0001/localmode-log-dir
2016-01-14 18:31:46,726 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F/.tez/application_1452796306690_0001/localmode-log-dir/history.txt.appattempt_1452796306690_0001_000000, maxErrors=10
2016-01-14 18:31:46,727 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:46,727 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796306690_0001_000000, appSubmitTime=1452796306713, launchTime=1452796306714
2016-01-14 18:31:46,731 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:58998
2016-01-14 18:31:46,731 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:46,732 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:46,732 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:46,733 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:46,733 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:46,735 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796306690_0001_000000, startTime=1452796306735
2016-01-14 18:31:46,735 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:46,735 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:46,812 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:46,812 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:46,812 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1452796306690_0001, dagName=[F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 18:31:46,827 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 18:31:46,828 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796306690_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F/.tez/application_1452796306690_0001/localmode-log-dir/dag_1452796306690_0001_1.dot
2016-01-14 18:31:46,828 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F/.tez/application_1452796306690_0001/localmode-log-dir/dag_1452796306690_0001_1-tez-dag.pb.txt
2016-01-14 18:31:46,835 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796306690_0001_1
2016-01-14 18:31:46,835 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:46,835 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 18:31:46 Running Dag: dag_1452796306690_0001_1
2016-01-14 18:31:46,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796306690_0001_1, submitTime=1452796306827
2016-01-14 18:31:46,844 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:46,844 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex B520CFDD3C404E3E9559E509BFD94835
2016-01-14 18:31:46,844 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:46,844 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796306690_0001_1, initTime=1452796306844
2016-01-14 18:31:46,845 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796306690_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:46,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:46,845 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:DAG_STARTED]: dagID=dag_1452796306690_0001_1, startTime=1452796306845
2016-01-14 18:31:46,845 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:46,845 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:46,845 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796306690_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:46,845 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C7CEFFCD095A4E0FB85C6404CEFF497B : {36758CB781F54628A2C5B2E8C1722259={InputName=36758CB781F54628A2C5B2E8C1722259}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:46,845 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 36758CB781F54628A2C5B2E8C1722259, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:46,845 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B]
2016-01-14 18:31:46,846 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B]
2016-01-14 18:31:46,846 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B]
2016-01-14 18:31:46,846 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B]. Starting root input initializers: 1
2016-01-14 18:31:46,846 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1452796306690_0001, dagName=[F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F] every/(1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 18:31:46,847 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:46,847 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835]
2016-01-14 18:31:46,847 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 36758CB781F54628A2C5B2E8C1722259 on vertex vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B]
2016-01-14 18:31:46,849 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:46,849 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 36758CB781F54628A2C5B2E8C1722259 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:46,849 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:46,849 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:46,850 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:46,850 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835]
2016-01-14 18:31:46,850 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835]
2016-01-14 18:31:46,851 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835]
2016-01-14 18:31:46,851 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=773A407B822E4D40A0786472447FE66B, vertexId=vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:46,851 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=773A407B822E4D40A0786472447FE66B, vertexId=vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835]
2016-01-14 18:31:46,852 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:46,852 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 36758CB781F54628A2C5B2E8C1722259 on vertex vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B]
2016-01-14 18:31:46,853 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=773A407B822E4D40A0786472447FE66B, vertexName=B520CFDD3C404E3E9559E509BFD94835, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:46,853 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=773A407B822E4D40A0786472447FE66B, vertexId=vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835]
2016-01-14 18:31:46,854 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:46,854 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:VERTEX_INITIALIZED]: vertexName=B520CFDD3C404E3E9559E509BFD94835, vertexId=vertex_1452796306690_0001_1_01, initRequestedTime=1452796306847, initedTime=1452796306854, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:46,855 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:46,855 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:46,855 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {36758CB781F54628A2C5B2E8C1722259=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:46,855 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B] parallelism set to 1
2016-01-14 18:31:46,855 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:46,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796306690_0001_1_00, eventCount=1
2016-01-14 18:31:46,856 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:46,856 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C7CEFFCD095A4E0FB85C6404CEFF497B, vertexId=vertex_1452796306690_0001_1_00, initRequestedTime=1452796306845, initedTime=1452796306856, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:46,856 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B] with distanceFromRoot: 0
2016-01-14 18:31:46,856 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:46,857 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C7CEFFCD095A4E0FB85C6404CEFF497B
2016-01-14 18:31:46,857 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B]
2016-01-14 18:31:46,857 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:46,857 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796306690_0001_1_00, startRequestedTime=1452796306856, startedTime=1452796306857
2016-01-14 18:31:46,857 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:46,858 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:46,858 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:TASK_STARTED]: vertexName=C7CEFFCD095A4E0FB85C6404CEFF497B, taskId=task_1452796306690_0001_1_00_000000, scheduledTime=1452796306858, launchTime=1452796306858
2016-01-14 18:31:46,858 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306690_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:46,858 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796306690_0001_1_00 for vertex: vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] numStartedSources: 1 numSources: 1
2016-01-14 18:31:46,858 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] with distanceFromRoot: 1
2016-01-14 18:31:46,858 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B]
2016-01-14 18:31:46,859 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C7CEFFCD095A4E0FB85C6404CEFF497B in vertex: B520CFDD3C404E3E9559E509BFD94835
2016-01-14 18:31:46,859 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: B520CFDD3C404E3E9559E509BFD94835 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:46,859 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:46,859 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796306690_0001_1_01, startRequestedTime=1452796306859, startedTime=1452796306859
2016-01-14 18:31:46,859 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:46,860 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: C7CEFFCD095A4E0FB85C6404CEFF497B, VertexParallelism: 1, TaskAttemptID:attempt_1452796306690_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=36758CB781F54628A2C5B2E8C1722259, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B520CFDD3C404E3E9559E509BFD94835, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:46,860 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306690_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:46,860 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:46,861 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:46,861 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:46,862 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:46,862 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796306690_0001_00_000001
2016-01-14 18:31:46,862 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:46,864 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796306690_0001_00_000001
2016-01-14 18:31:46,864 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:46,865 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:46,865 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796306690_0001_00_000001, launchTime=1452796306865
2016-01-14 18:31:46,865 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:46,866 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796306690_0001_00_000001
2016-01-14 18:31:46,866 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796306690_0001_1_00_000000_0] to container: [container_1452796306690_0001_00_000001]
2016-01-14 18:31:46,866 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:46,867 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796306690_0001_00_000001 given task: attempt_1452796306690_0001_1_00_000000_0
2016-01-14 18:31:46,867 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796306690_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796306690_0001_1_00_000000_0
2016-01-14 18:31:46,867 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796306690_0001_1_00_000000_0] started. Is using containerId: [container_1452796306690_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:46,867 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:46,867 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C7CEFFCD095A4E0FB85C6404CEFF497B, taskAttemptId=attempt_1452796306690_0001_1_00_000000_0, startTime=1452796306867, containerId=container_1452796306690_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796306690_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:46,867 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306690_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:46,867 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306690_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:46,868 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796306690_0001, with dag id: dag_1452796306690_0001_1
2016-01-14 18:31:46 Starting to run new task attempt: attempt_1452796306690_0001_1_00_000000_0
2016-01-14 18:31:46,868 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796306690_0001_1_00_000000_0
2016-01-14 18:31:46,868 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:46,868 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:46,868 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:46,868 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:46,868 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:46,869 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:46,869 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: C7CEFFCD095A4E0FB85C6404CEFF497B, VertexParallelism: 1, TaskAttemptID:attempt_1452796306690_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=36758CB781F54628A2C5B2E8C1722259, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=B520CFDD3C404E3E9559E509BFD94835, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:46,869 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:46,870 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796306690_0001_1_00_000000_0
2016-01-14 18:31:46,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:46,870 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=36758CB781F54628A2C5B2E8C1722259, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:46,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=B520CFDD3C404E3E9559E509BFD94835, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:46,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,871 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796306690_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:46,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:46,872 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: B520CFDD3C404E3E9559E509BFD94835
2016-01-14 18:31:46,871 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:46,874 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 36758CB781F54628A2C5B2E8C1722259
2016-01-14 18:31:46,874 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 36758CB781F54628A2C5B2E8C1722259
2016-01-14 18:31:46,874 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:46,876 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:46,876 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 36758CB781F54628A2C5B2E8C1722259
2016-01-14 18:31:46,876 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 36758CB781F54628A2C5B2E8C1722259
2016-01-14 18:31:46,877 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:46,877 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: B520CFDD3C404E3E9559E509BFD94835
2016-01-14 18:31:46,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: B520CFDD3C404E3E9559E509BFD94835
2016-01-14 18:31:46,877 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C7CEFFCD095A4E0FB85C6404CEFF497B, ordinal: 0
2016-01-14 18:31:46,877 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7CEFFCD095A4E0FB85C6404CEFF497B, mem on start (mb), free: 168, total: 564, max: 672
2016-01-14 18:31:46,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,877 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:46,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:46,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:46,878 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:46,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:46,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:46,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:46,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:46,878 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:46,878 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 36758CB781F54628A2C5B2E8C1722259, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:46,878 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, B520CFDD3C404E3E9559E509BFD94835, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:46,878 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:46,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 36758CB781F54628A2C5B2E8C1722259 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:46,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:46,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:46,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 36758CB781F54628A2C5B2E8C1722259
2016-01-14 18:31:46,879 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:46,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 36758CB781F54628A2C5B2E8C1722259
2016-01-14 18:31:46,879 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:46,879 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796306690_0001_1_00_000000_0
2016-01-14 18:31:46,880 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:46,880 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@3214c136
2016-01-14 18:31:46,881 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@3214c136
2016-01-14 18:31:46,881 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:46,881 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:46,882 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:46,883 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 36758CB781F54628A2C5B2E8C1722259
2016-01-14 18:31:46,883 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: C134031D20354AD9A5EE69194A590ABF
2016-01-14 18:31:46,883 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] C134031D20354AD9A5EE69194A590ABF
2016-01-14 18:31:46,883 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@172e92d7; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@72aa6d24; comparator=cascading.tuple.hadoop.util.TupleComparator@12e3914c; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:46,884 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:46,884 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:46,884 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:46,906 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:46,906 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:46,906 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:46,906 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:46,906 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:46,907 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:46,907 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:46,907 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C7CEFFCD095A4E0FB85C6404CEFF497B, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:46,910 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796306690_0001_1_00_000000_0
2016-01-14 18:31:46,910 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C7CEFFCD095A4E0FB85C6404CEFF497B, mem on close (mb), free: 280, total: 564, max: 672
2016-01-14 18:31:46,910 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:46,910 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:46,910 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:46,910 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:46,912 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:46 Completed running task attempt: attempt_1452796306690_0001_1_00_000000_0
2016-01-14 18:31:46,914 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796306690_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:46,914 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796306908, Process_Duration=2, Process_End_Time=1452796306910, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:46,914 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796306690_0001_1_00, eventCount=1
2016-01-14 18:31:46,915 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C7CEFFCD095A4E0FB85C6404CEFF497B, taskAttemptId=attempt_1452796306690_0001_1_00_000000_0, startTime=1452796306867, finishTime=1452796306915, timeTaken=48, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796306908, Process_Duration=2, Process_End_Time=1452796306910, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:46,915 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:46,915 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:46,915 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:46,915 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306690_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:46,916 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796306690_0001_00_000001
2016-01-14 18:31:46,916 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796306690_0001_1_00_000000_0
2016-01-14 18:31:46,916 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:46,916 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:46,916 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:TASK_FINISHED]: vertexName=C7CEFFCD095A4E0FB85C6404CEFF497B, taskId=task_1452796306690_0001_1_00_000000, startTime=1452796306867, finishTime=1452796306916, timeTaken=49, status=SUCCEEDED, successfulAttemptID=attempt_1452796306690_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796306908, Process_Duration=2, Process_End_Time=1452796306910, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:46,916 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306690_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:46,916 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B] : 1
2016-01-14 18:31:46,916 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B]
2016-01-14 18:31:46,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:VERTEX_FINISHED]: vertexName=C7CEFFCD095A4E0FB85C6404CEFF497B, vertexId=vertex_1452796306690_0001_1_00, initRequestedTime=1452796306845, initedTime=1452796306856, startRequestedTime=1452796306856, startedTime=1452796306857, finishTime=1452796306916, timeTaken=59, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796306908, Process_Duration=2, Process_End_Time=1452796306910, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796306867, firstTasksToStart=[ task_1452796306690_0001_1_00_000000 ], lastTaskFinishTime=1452796306915, lastTasksToFinish=[ task_1452796306690_0001_1_00_000000 ], minTaskDuration=48, maxTaskDuration=48, avgTaskDuration=48.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796306690_0001_1_00_000000 ], longestDurationTasks=[ task_1452796306690_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:46,917 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:46,917 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:46,917 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796306690_0001_00_000001, stoppedTime=1452796306917, exitStatus=0
2016-01-14 18:31:46,918 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:46,918 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] attempt: attempt_1452796306690_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:46,918 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: B520CFDD3C404E3E9559E509BFD94835
2016-01-14 18:31:46,918 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835]
2016-01-14 18:31:46,918 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:46,918 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796306690_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:46,918 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:TASK_STARTED]: vertexName=B520CFDD3C404E3E9559E509BFD94835, taskId=task_1452796306690_0001_1_01_000000, scheduledTime=1452796306918, launchTime=1452796306918
2016-01-14 18:31:46,918 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306690_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:46,918 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835]
2016-01-14 18:31:46,919 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:46,919 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: B520CFDD3C404E3E9559E509BFD94835, VertexParallelism: 1, TaskAttemptID:attempt_1452796306690_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C7CEFFCD095A4E0FB85C6404CEFF497B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=773A407B822E4D40A0786472447FE66B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:46,919 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306690_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:46,919 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:46,919 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:46,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:46,920 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:46,920 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796306690_0001_00_000002
2016-01-14 18:31:46,921 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:46,923 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796306690_0001_00_000002
2016-01-14 18:31:46,923 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:46,923 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:46,923 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796306690_0001_00_000002, launchTime=1452796306923
2016-01-14 18:31:46,923 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:46,924 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796306690_0001_00_000002
2016-01-14 18:31:46,925 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796306690_0001_1_01_000000_0] to container: [container_1452796306690_0001_00_000002]
2016-01-14 18:31:46,925 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:46,925 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796306690_0001_00_000002 given task: attempt_1452796306690_0001_1_01_000000_0
2016-01-14 18:31:46,925 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796306690_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796306690_0001_1_01_000000_0
2016-01-14 18:31:46,925 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796306690_0001_1_01_000000_0] started. Is using containerId: [container_1452796306690_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:46 Starting to run new task attempt: attempt_1452796306690_0001_1_01_000000_0
2016-01-14 18:31:46,925 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:46,925 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796306690_0001_1_01_000000_0
2016-01-14 18:31:46,926 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:46,926 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:46,926 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:46,926 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:46,926 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:46,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=B520CFDD3C404E3E9559E509BFD94835, taskAttemptId=attempt_1452796306690_0001_1_01_000000_0, startTime=1452796306925, containerId=container_1452796306690_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796306690_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:46,926 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306690_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:46,926 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306690_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:46,926 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796306690_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:46,926 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:46,927 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F] every/(1/1) ...platform/ALL_ARGS_RESULTS, VertexName: B520CFDD3C404E3E9559E509BFD94835, VertexParallelism: 1, TaskAttemptID:attempt_1452796306690_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C7CEFFCD095A4E0FB85C6404CEFF497B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=773A407B822E4D40A0786472447FE66B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:46,927 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796306690_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:46,927 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:46,927 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796306690_0001_00_000001, Exiting
2016-01-14 18:31:46,927 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796306690_0001_00_000001
2016-01-14 18:31:46,927 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796306690_0001_00_000001
2016-01-14 18:31:46,927 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796306690_0001_1_01_000000_0
2016-01-14 18:31:46,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:46,928 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796306690_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:46,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,928 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796306690_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:46,928 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796306690_0001_00_000001 completed successfully
2016-01-14 18:31:46,929 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:46,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C7CEFFCD095A4E0FB85C6404CEFF497B, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:46,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:46,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=773A407B822E4D40A0786472447FE66B, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:46,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C7CEFFCD095A4E0FB85C6404CEFF497B
2016-01-14 18:31:46,929 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:46,930 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 773A407B822E4D40A0786472447FE66B
2016-01-14 18:31:46,930 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:46,930 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 773A407B822E4D40A0786472447FE66B
2016-01-14 18:31:46,931 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:46,931 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:46,931 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C7CEFFCD095A4E0FB85C6404CEFF497B
2016-01-14 18:31:46,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C7CEFFCD095A4E0FB85C6404CEFF497B
2016-01-14 18:31:46,934 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:46,937 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: B520CFDD3C404E3E9559E509BFD94835, ordinal: 1
2016-01-14 18:31:46,937 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B520CFDD3C404E3E9559E509BFD94835, mem on start (mb), free: 274, total: 564, max: 672
2016-01-14 18:31:46,937 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:46,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:46,938 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:46,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 773A407B822E4D40A0786472447FE66B
2016-01-14 18:31:46,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:46,938 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:46,938 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:46,938 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:46,938 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:46,939 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:46,939 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:46,939 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:46,939 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 773A407B822E4D40A0786472447FE66B, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:46,939 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C7CEFFCD095A4E0FB85C6404CEFF497B, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:46,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:46,939 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C7CEFFCD095A4E0FB85C6404CEFF497B
2016-01-14 18:31:46,940 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:46,940 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:46,940 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C7CEFFCD095A4E0FB85C6404CEFF497B with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:46,940 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C7CEFFCD095A4E0FB85C6404CEFF497B being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:46,940 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:46,940 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:46,941 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:46,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:46,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:46,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C7CEFFCD095A4E0FB85C6404CEFF497B
2016-01-14 18:31:46,941 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:46,942 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796306690_0001_1_01_000000_0
2016-01-14 18:31:46,942 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796306690_0001_1_00_000000_0_10327, runDuration: 40275, ]
2016-01-14 18:31:46,943 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796306690_0001_1_00_000000_0_10327] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796306943, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:46,943 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C7CEFFCD095A4E0FB85C6404CEFF497B
2016-01-14 18:31:46,943 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:46,945 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:46,945 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C7CEFFCD095A4E0FB85C6404CEFF497B] #55 in 3ms
2016-01-14 18:31:46,945 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C7CEFFCD095A4E0FB85C6404CEFF497B] #55, status:false, isInterrupted:false
2016-01-14 18:31:46,946 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:46,946 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:46,946 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:46,946 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:46,946 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: C134031D20354AD9A5EE69194A590ABF
2016-01-14 18:31:46,946 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:46,947 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C7CEFFCD095A4E0FB85C6404CEFF497B
2016-01-14 18:31:46,947 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:46,947 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS"], id: 773A407B822E4D40A0786472447FE66B
2016-01-14 18:31:46,947 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_RESULTS"]
2016-01-14 18:31:46,947 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] C134031D20354AD9A5EE69194A590ABF, for 1 inputs
2016-01-14 18:31:46,947 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: B520CFDD3C404E3E9559E509BFD94835, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:46,948 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@3c48418a; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@2bc6e9f1
2016-01-14 18:31:46,948 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:46,951 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:46,953 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:46,954 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:46,954 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796306690_0001_1_01_000000_0
2016-01-14 18:31:46,954 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796306690_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:46,955 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796306690_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:46,958 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=B520CFDD3C404E3E9559E509BFD94835, taskAttemptId=attempt_1452796306690_0001_1_01_000000_0, startTime=1452796306925, finishTime=1452796306958, timeTaken=33, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796306948, Process_Duration=7, Process_End_Time=1452796306955, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:46,958 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:46,959 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306690_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:46,959 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796306690_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:46,959 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:TASK_FINISHED]: vertexName=B520CFDD3C404E3E9559E509BFD94835, taskId=task_1452796306690_0001_1_01_000000, startTime=1452796306925, finishTime=1452796306959, timeTaken=34, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:46,959 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796306690_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:46,959 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:46,959 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] : 1
2016-01-14 18:31:46,959 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] because task failed: task_1452796306690_0001_1_01_000000
2016-01-14 18:31:46,959 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:46,960 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:46,960 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:46,960 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835]
2016-01-14 18:31:46,960 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=773A407B822E4D40A0786472447FE66B, vertexId=vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835]
2016-01-14 18:31:46,961 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:VERTEX_FINISHED]: vertexName=B520CFDD3C404E3E9559E509BFD94835, vertexId=vertex_1452796306690_0001_1_01, initRequestedTime=1452796306847, initedTime=1452796306854, startRequestedTime=1452796306859, startedTime=1452796306859, finishTime=1452796306961, timeTaken=102, status=FAILED, diagnostics=Task failed, taskId=task_1452796306690_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796306925, firstTasksToStart=[ task_1452796306690_0001_1_01_000000 ], lastTaskFinishTime=1452796306958, lastTasksToFinish=[ task_1452796306690_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:46,961 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:46,961 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:46,962 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796306690_0001_00_000002, stoppedTime=1452796306962, exitStatus=0
2016-01-14 18:31:46,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=3, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796306948, Process_Duration=7, Process_End_Time=1452796306955, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:46,962 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: B520CFDD3C404E3E9559E509BFD94835, mem on close (mb), free: 270, total: 564, max: 672
2016-01-14 18:31:46,962 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=B520CFDD3C404E3E9559E509BFD94835, index=1
2016-01-14 18:31:46,962 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:46,962 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:46,963 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796306690_0001_1_01_000000_0 failed: true container: container_1452796306690_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:46,962 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C7CEFFCD095A4E0FB85C6404CEFF497B
2016-01-14 18:31:46,963 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:46,963 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:46,963 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:46,963 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:46,963 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:46,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=B520CFDD3C404E3E9559E509BFD94835, sourceVertex=C7CEFFCD095A4E0FB85C6404CEFF497B
2016-01-14 18:31:46,963 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=B520CFDD3C404E3E9559E509BFD94835, sourceVertex=773A407B822E4D40A0786472447FE66B
2016-01-14 18:31:46,963 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:46,964 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:46,964 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796306690_0001_1
2016-01-14 18:31:46,964 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796306690_0001_00_000002
2016-01-14 18:31:46,964 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796306690_0001_00_000002
2016-01-14 18:31:46,964 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796306690_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:46,964 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796306690_0001_00_000002 completed but with errors
2016-01-14 18:31:46,965 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 773A407B822E4D40A0786472447FE66B for vertex: vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835]
2016-01-14 18:31:46,965 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796306690_0001_1_00 [C7CEFFCD095A4E0FB85C6404CEFF497B]
2016-01-14 18:31:46,965 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796306690_0001_1, queueSize=0
2016-01-14 18:31:46,966 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:46,966 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796306690_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796306690_0001_1, startTime=1452796306845, finishTime=1452796306965, timeTaken=120, status=FAILED, diagnostics=Vertex failed, vertexName=B520CFDD3C404E3E9559E509BFD94835, vertexId=vertex_1452796306690_0001_1_01, diagnostics=[Task failed, taskId=task_1452796306690_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796306690_0001_1_01 [B520CFDD3C404E3E9559E509BFD94835] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=8, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796306908, Process_Duration=2, Process_End_Time=1452796306910, Read_Duration=1, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:46,966 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796306690_0001_1 finished with state: FAILED
2016-01-14 18:31:46,966 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796306690_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:46,966 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796306690_0001_00_000002
2016-01-14 18:31:46,966 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796306690_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:46,966 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796306690_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:46 Completed Dag: dag_1452796306690_0001_1
2016-01-14 18:31:46,967 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796306690_0001_1, dagState=FAILED
2016-01-14 18:31:46,967 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796306690_0001_1_post
2016-01-14 18:31:46,967 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:46,967 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:46,967 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796306690_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:46,973 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_ARGS_RESULTS, applicationId=application_1452796306690_0001
2016-01-14 18:31:46,975 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:46,975 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:46,975 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:46,975 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:46,975 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:46,976 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:46,976 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:46,976 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:46,976 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:46,977 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:46,977 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:46,977 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:46,977 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:46,980 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/F7592442883D4772879A5E6436B24A7B/A7E8552F6BAA4FBF9DA197A5E05A1A2F/.tez/application_1452796306690_0001
2016-01-14 18:31:46,980 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:46,981 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...platform/ALL_ARGS_RESULTS
2016-01-14 18:31:46,981 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:46,981 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.309
2016-01-14 18:31:46,981 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_RESULTS
2016-01-14 18:31:47,003 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:47,003 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:47,033 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.004
2016-01-14 18:31:47,035 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: ILLEGAL, in: 00:00.007
2016-01-14 18:31:47,035 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:47,035 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, found assembly to be malformed
2016-01-14 18:31:47,035 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_ALL
2016-01-14 18:31:47,067 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:47,067 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:47,093 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:47,093 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:47,099 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 18:31:47,105 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.015
2016-01-14 18:31:47,105 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:47,106 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:47,106 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:47,127 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:47,134 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE
2016-01-14 18:31:47,195 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:47,196 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:47,196 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE"]
2016-01-14 18:31:47,196 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:47,196 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:47,196 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:47,197 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 18:31:47,197 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:47,208 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:47,212 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:47,213 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:47,213 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:47,222 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476/.tez/application_1452796307212_0001 doesn't exist and is created
2016-01-14 18:31:47,239 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:47,239 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:47,239 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476/.tez/application_1452796307212_0001
2016-01-14 18:31:47,240 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796307212_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:47,250 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:47,251 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:47,252 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:47,252 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:47,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476/.tez/application_1452796307212_0001/localmode-log-dir
2016-01-14 18:31:47,252 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476/.tez/application_1452796307212_0001/localmode-log-dir/history.txt.appattempt_1452796307212_0001_000000, maxErrors=10
2016-01-14 18:31:47,252 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:47,253 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796307212_0001_000000, appSubmitTime=1452796307240, launchTime=1452796307240
2016-01-14 18:31:47,256 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:47,258 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:47,258 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:35834
2016-01-14 18:31:47,258 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:47,260 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:47,262 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796307212_0001_000000, startTime=1452796307262
2016-01-14 18:31:47,262 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:47,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:47,264 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:47,339 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:47,339 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:47,339 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1452796307212_0001, dagName=[19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 18:31:47,354 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 18:31:47,355 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796307212_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476/.tez/application_1452796307212_0001/localmode-log-dir/dag_1452796307212_0001_1.dot
2016-01-14 18:31:47,356 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476/.tez/application_1452796307212_0001/localmode-log-dir/dag_1452796307212_0001_1-tez-dag.pb.txt
2016-01-14 18:31:47,362 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796307212_0001_1
2016-01-14 18:31:47,362 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:47,362 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 18:31:47 Running Dag: dag_1452796307212_0001_1
2016-01-14 18:31:47,371 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:47,371 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796307212_0001_1, submitTime=1452796307354
2016-01-14 18:31:47,372 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex CF832548685B4815BA5DD7D49DDC03B7
2016-01-14 18:31:47,372 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:47,372 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:47,372 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796307212_0001_1, initTime=1452796307371
2016-01-14 18:31:47,373 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796307212_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:47,373 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:DAG_STARTED]: dagID=dag_1452796307212_0001_1, startTime=1452796307373
2016-01-14 18:31:47,373 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:47,373 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796307212_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:47,373 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: D1F254F4F9E242DCBD6DF8997EE5479C : {1861695DCC8B4AFA967A7959AD8733E2={InputName=1861695DCC8B4AFA967A7959AD8733E2}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:47,373 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 1861695DCC8B4AFA967A7959AD8733E2, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:47,373 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C]
2016-01-14 18:31:47,373 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:47,373 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C]
2016-01-14 18:31:47,373 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C]
2016-01-14 18:31:47,374 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C]. Starting root input initializers: 1
2016-01-14 18:31:47,374 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:47,374 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7]
2016-01-14 18:31:47,375 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1452796307212_0001, dagName=[19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476] every/(1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 18:31:47,376 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:47,376 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7]
2016-01-14 18:31:47,377 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7]
2016-01-14 18:31:47,377 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7]
2016-01-14 18:31:47,377 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=9988A86CCFC04A42B9BE5DAA07FB70E2, vertexId=vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:47,377 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=9988A86CCFC04A42B9BE5DAA07FB70E2, vertexId=vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7]
2016-01-14 18:31:47,378 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 1861695DCC8B4AFA967A7959AD8733E2 on vertex vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C]
2016-01-14 18:31:47,379 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=9988A86CCFC04A42B9BE5DAA07FB70E2, vertexName=CF832548685B4815BA5DD7D49DDC03B7, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:47,379 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=9988A86CCFC04A42B9BE5DAA07FB70E2, vertexId=vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7]
2016-01-14 18:31:47,380 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:47,380 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:VERTEX_INITIALIZED]: vertexName=CF832548685B4815BA5DD7D49DDC03B7, vertexId=vertex_1452796307212_0001_1_01, initRequestedTime=1452796307374, initedTime=1452796307379, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:47,380 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:47,380 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:47,381 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 1861695DCC8B4AFA967A7959AD8733E2 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:47,381 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:47,381 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:47,382 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:47,382 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 1861695DCC8B4AFA967A7959AD8733E2 on vertex vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C]
2016-01-14 18:31:47,383 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:47,383 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {1861695DCC8B4AFA967A7959AD8733E2=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:47,383 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C] parallelism set to 1
2016-01-14 18:31:47,383 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:47,383 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796307212_0001_1_00, eventCount=1
2016-01-14 18:31:47,384 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:47,384 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D1F254F4F9E242DCBD6DF8997EE5479C, vertexId=vertex_1452796307212_0001_1_00, initRequestedTime=1452796307373, initedTime=1452796307384, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:47,384 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C] with distanceFromRoot: 0
2016-01-14 18:31:47,384 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:47,384 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in D1F254F4F9E242DCBD6DF8997EE5479C
2016-01-14 18:31:47,385 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C]
2016-01-14 18:31:47,385 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:47,385 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796307212_0001_1_00, startRequestedTime=1452796307384, startedTime=1452796307384
2016-01-14 18:31:47,385 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:47,385 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:47,386 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:TASK_STARTED]: vertexName=D1F254F4F9E242DCBD6DF8997EE5479C, taskId=task_1452796307212_0001_1_00_000000, scheduledTime=1452796307385, launchTime=1452796307385
2016-01-14 18:31:47,386 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307212_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:47,386 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796307212_0001_1_00 for vertex: vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] numStartedSources: 1 numSources: 1
2016-01-14 18:31:47,386 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] with distanceFromRoot: 1
2016-01-14 18:31:47,386 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C]
2016-01-14 18:31:47,387 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: D1F254F4F9E242DCBD6DF8997EE5479C in vertex: CF832548685B4815BA5DD7D49DDC03B7
2016-01-14 18:31:47,387 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: CF832548685B4815BA5DD7D49DDC03B7 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:47,387 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:47,387 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796307212_0001_1_01, startRequestedTime=1452796307387, startedTime=1452796307387
2016-01-14 18:31:47,387 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:47,388 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: D1F254F4F9E242DCBD6DF8997EE5479C, VertexParallelism: 1, TaskAttemptID:attempt_1452796307212_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1861695DCC8B4AFA967A7959AD8733E2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CF832548685B4815BA5DD7D49DDC03B7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:47,388 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307212_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:47,388 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:47,388 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:47,389 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:47,389 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:47,389 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796307212_0001_00_000001
2016-01-14 18:31:47,389 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:47,391 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796307212_0001_00_000001
2016-01-14 18:31:47,391 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:47,392 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:47,392 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:47,392 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796307212_0001_00_000001, launchTime=1452796307392
2016-01-14 18:31:47,393 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796307212_0001_00_000001
2016-01-14 18:31:47,393 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796307212_0001_1_00_000000_0] to container: [container_1452796307212_0001_00_000001]
2016-01-14 18:31:47,393 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:47,393 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796307212_0001_00_000001 given task: attempt_1452796307212_0001_1_00_000000_0
2016-01-14 18:31:47,393 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796307212_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796307212_0001_1_00_000000_0
2016-01-14 18:31:47,394 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796307212_0001_1_00_000000_0] started. Is using containerId: [container_1452796307212_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:47,394 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:47 Starting to run new task attempt: attempt_1452796307212_0001_1_00_000000_0
2016-01-14 18:31:47,394 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796307212_0001_1_00_000000_0
2016-01-14 18:31:47,394 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:47,394 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D1F254F4F9E242DCBD6DF8997EE5479C, taskAttemptId=attempt_1452796307212_0001_1_00_000000_0, startTime=1452796307393, containerId=container_1452796307212_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796307212_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:47,394 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:47,394 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:47,394 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:47,394 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:47,394 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307212_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:47,394 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:47,394 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307212_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:47,394 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: D1F254F4F9E242DCBD6DF8997EE5479C, VertexParallelism: 1, TaskAttemptID:attempt_1452796307212_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=1861695DCC8B4AFA967A7959AD8733E2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=CF832548685B4815BA5DD7D49DDC03B7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:47,395 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:47,395 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796307212_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:47,397 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796307212_0001_1_00_000000_0
2016-01-14 18:31:47,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:47,397 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,397 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796307212_0001, with dag id: dag_1452796307212_0001_1
2016-01-14 18:31:47,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,398 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=1861695DCC8B4AFA967A7959AD8733E2, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:47,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:47,399 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 1861695DCC8B4AFA967A7959AD8733E2
2016-01-14 18:31:47,399 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 1861695DCC8B4AFA967A7959AD8733E2
2016-01-14 18:31:47,401 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:47,401 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:47,401 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 1861695DCC8B4AFA967A7959AD8733E2
2016-01-14 18:31:47,401 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 1861695DCC8B4AFA967A7959AD8733E2
2016-01-14 18:31:47,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=CF832548685B4815BA5DD7D49DDC03B7, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:47,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:47,403 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: CF832548685B4815BA5DD7D49DDC03B7
2016-01-14 18:31:47,404 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D1F254F4F9E242DCBD6DF8997EE5479C, ordinal: 0
2016-01-14 18:31:47,404 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1F254F4F9E242DCBD6DF8997EE5479C, mem on start (mb), free: 165, total: 564, max: 672
2016-01-14 18:31:47,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:47,404 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:47,404 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:47,405 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: CF832548685B4815BA5DD7D49DDC03B7
2016-01-14 18:31:47,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: CF832548685B4815BA5DD7D49DDC03B7
2016-01-14 18:31:47,405 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:47,405 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:47,406 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:47,406 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:47,406 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:47,406 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:47,407 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:47,407 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 1861695DCC8B4AFA967A7959AD8733E2, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:47,407 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, CF832548685B4815BA5DD7D49DDC03B7, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:47,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:47,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 1861695DCC8B4AFA967A7959AD8733E2 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:47,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:47,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 1861695DCC8B4AFA967A7959AD8733E2
2016-01-14 18:31:47,407 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:47,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 1861695DCC8B4AFA967A7959AD8733E2
2016-01-14 18:31:47,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:47,407 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:47,408 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796307212_0001_1_00_000000_0
2016-01-14 18:31:47,408 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:47,408 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@32a00cd4
2016-01-14 18:31:47,409 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@32a00cd4
2016-01-14 18:31:47,409 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:47,409 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:47,410 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:47,411 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 1861695DCC8B4AFA967A7959AD8733E2
2016-01-14 18:31:47,411 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 0D55A67ECDEF4C46887240430214B650
2016-01-14 18:31:47,411 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 0D55A67ECDEF4C46887240430214B650
2016-01-14 18:31:47,411 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@2f424d41; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@29ddfea4; comparator=cascading.tuple.hadoop.util.TupleComparator@2d78d675; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:47,411 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:47,412 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:47,412 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:47,436 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:47,436 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:47,436 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:47,436 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:47,436 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:47,437 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:47,437 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:47,437 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D1F254F4F9E242DCBD6DF8997EE5479C, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:47,439 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796307212_0001_1_00_000000_0
2016-01-14 18:31:47,439 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D1F254F4F9E242DCBD6DF8997EE5479C, mem on close (mb), free: 277, total: 564, max: 672
2016-01-14 18:31:47,439 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:47,439 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:47,439 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:47,439 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:47,441 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:47 Completed running task attempt: attempt_1452796307212_0001_1_00_000000_0
2016-01-14 18:31:47,444 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796307212_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:47,444 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796307437, Process_Duration=2, Process_End_Time=1452796307439, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:47,444 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796307212_0001_1_00, eventCount=1
2016-01-14 18:31:47,445 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:47,445 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:47,445 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796307212_0001_00_000001
2016-01-14 18:31:47,445 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D1F254F4F9E242DCBD6DF8997EE5479C, taskAttemptId=attempt_1452796307212_0001_1_00_000000_0, startTime=1452796307393, finishTime=1452796307445, timeTaken=52, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796307437, Process_Duration=2, Process_End_Time=1452796307439, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:47,445 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307212_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:47,445 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:47,445 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:47,445 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796307212_0001_1_00_000000_0
2016-01-14 18:31:47,446 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:TASK_FINISHED]: vertexName=D1F254F4F9E242DCBD6DF8997EE5479C, taskId=task_1452796307212_0001_1_00_000000, startTime=1452796307393, finishTime=1452796307445, timeTaken=52, status=SUCCEEDED, successfulAttemptID=attempt_1452796307212_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796307437, Process_Duration=2, Process_End_Time=1452796307439, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:47,446 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307212_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:47,446 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C] : 1
2016-01-14 18:31:47,446 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C]
2016-01-14 18:31:47,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:VERTEX_FINISHED]: vertexName=D1F254F4F9E242DCBD6DF8997EE5479C, vertexId=vertex_1452796307212_0001_1_00, initRequestedTime=1452796307373, initedTime=1452796307384, startRequestedTime=1452796307384, startedTime=1452796307384, finishTime=1452796307446, timeTaken=62, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796307437, Process_Duration=2, Process_End_Time=1452796307439, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796307393, firstTasksToStart=[ task_1452796307212_0001_1_00_000000 ], lastTaskFinishTime=1452796307445, lastTasksToFinish=[ task_1452796307212_0001_1_00_000000 ], minTaskDuration=52, maxTaskDuration=52, avgTaskDuration=52.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796307212_0001_1_00_000000 ], longestDurationTasks=[ task_1452796307212_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:47,447 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:47,447 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:47,447 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796307212_0001_00_000001, stoppedTime=1452796307447, exitStatus=0
2016-01-14 18:31:47,447 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:47,447 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] attempt: attempt_1452796307212_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:47,447 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: CF832548685B4815BA5DD7D49DDC03B7
2016-01-14 18:31:47,448 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7]
2016-01-14 18:31:47,448 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:47,448 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796307212_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:47,448 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:TASK_STARTED]: vertexName=CF832548685B4815BA5DD7D49DDC03B7, taskId=task_1452796307212_0001_1_01_000000, scheduledTime=1452796307448, launchTime=1452796307448
2016-01-14 18:31:47,448 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307212_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:47,448 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:47,448 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7]
2016-01-14 18:31:47,448 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: CF832548685B4815BA5DD7D49DDC03B7, VertexParallelism: 1, TaskAttemptID:attempt_1452796307212_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D1F254F4F9E242DCBD6DF8997EE5479C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9988A86CCFC04A42B9BE5DAA07FB70E2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:47,449 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307212_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:47,449 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:47,449 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:47,449 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:47,450 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:47,451 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:47,452 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796307212_0001_00_000002
2016-01-14 18:31:47,452 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:47,454 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796307212_0001_00_000002
2016-01-14 18:31:47,454 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:47,454 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:47,454 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:47,455 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796307212_0001_00_000002, launchTime=1452796307454
2016-01-14 18:31:47,455 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796307212_0001_00_000002
2016-01-14 18:31:47,455 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796307212_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:47,455 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796307212_0001_00_000001: 10 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:47,456 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796307212_0001_00_000001, Exiting
2016-01-14 18:31:47,456 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796307212_0001_00_000001
2016-01-14 18:31:47,456 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796307212_0001_00_000001
2016-01-14 18:31:47,456 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796307212_0001_1_01_000000_0] to container: [container_1452796307212_0001_00_000002]
2016-01-14 18:31:47,456 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:47,456 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796307212_0001_1_01_000000_0] started. Is using containerId: [container_1452796307212_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:47,456 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:47,456 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796307212_0001_00_000002 given task: attempt_1452796307212_0001_1_01_000000_0
2016-01-14 18:31:47,456 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=CF832548685B4815BA5DD7D49DDC03B7, taskAttemptId=attempt_1452796307212_0001_1_01_000000_0, startTime=1452796307456, containerId=container_1452796307212_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796307212_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:47,457 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307212_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:47,457 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307212_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:47,457 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796307212_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796307212_0001_1_01_000000_0
2016-01-14 18:31:47,457 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796307212_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:47,458 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796307212_0001_00_000001 completed successfully
2016-01-14 18:31:47 Starting to run new task attempt: attempt_1452796307212_0001_1_01_000000_0
2016-01-14 18:31:47,458 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796307212_0001_1_01_000000_0
2016-01-14 18:31:47,458 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:47,458 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:47,458 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:47,458 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:47,458 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:47,458 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:47,458 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:47,458 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476] every/(1/1) ...platform/ALL_ARGS_REPLACE, VertexName: CF832548685B4815BA5DD7D49DDC03B7, VertexParallelism: 1, TaskAttemptID:attempt_1452796307212_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=D1F254F4F9E242DCBD6DF8997EE5479C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=9988A86CCFC04A42B9BE5DAA07FB70E2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:47,459 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:47,459 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796307212_0001_1_01_000000_0
2016-01-14 18:31:47,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:47,459 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,460 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796307212_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:47,460 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=D1F254F4F9E242DCBD6DF8997EE5479C, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:47,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:47,461 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: D1F254F4F9E242DCBD6DF8997EE5479C
2016-01-14 18:31:47,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=9988A86CCFC04A42B9BE5DAA07FB70E2, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:47,462 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:47,463 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 9988A86CCFC04A42B9BE5DAA07FB70E2
2016-01-14 18:31:47,463 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:47,463 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 9988A86CCFC04A42B9BE5DAA07FB70E2
2016-01-14 18:31:47,465 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:47,466 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:47,466 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:47,466 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: D1F254F4F9E242DCBD6DF8997EE5479C
2016-01-14 18:31:47,466 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: D1F254F4F9E242DCBD6DF8997EE5479C
2016-01-14 18:31:47,469 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: CF832548685B4815BA5DD7D49DDC03B7, ordinal: 1
2016-01-14 18:31:47,469 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CF832548685B4815BA5DD7D49DDC03B7, mem on start (mb), free: 271, total: 564, max: 672
2016-01-14 18:31:47,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:47,469 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:47,471 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:47,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 9988A86CCFC04A42B9BE5DAA07FB70E2
2016-01-14 18:31:47,471 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:47,471 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:47,471 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:47,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:47,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:47,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:47,472 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:47,472 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 9988A86CCFC04A42B9BE5DAA07FB70E2, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:47,472 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, D1F254F4F9E242DCBD6DF8997EE5479C, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:47,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:47,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: D1F254F4F9E242DCBD6DF8997EE5479C being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:47,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:47,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: D1F254F4F9E242DCBD6DF8997EE5479C
2016-01-14 18:31:47,472 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:47,473 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:47,473 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:47,473 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: D1F254F4F9E242DCBD6DF8997EE5479C with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:47,473 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:47,473 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:47,474 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:47,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: D1F254F4F9E242DCBD6DF8997EE5479C
2016-01-14 18:31:47,474 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:47,474 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796307212_0001_1_01_000000_0
2016-01-14 18:31:47,476 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796307212_0001_1_00_000000_0_10333, runDuration: 39471, ]
2016-01-14 18:31:47,477 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796307212_0001_1_00_000000_0_10333] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796307477, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:47,477 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : D1F254F4F9E242DCBD6DF8997EE5479C
2016-01-14 18:31:47,477 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:47,478 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:47,478 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [D1F254F4F9E242DCBD6DF8997EE5479C] #56 in 2ms
2016-01-14 18:31:47,478 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [D1F254F4F9E242DCBD6DF8997EE5479C] #56, status:false, isInterrupted:false
2016-01-14 18:31:47,479 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 0D55A67ECDEF4C46887240430214B650
2016-01-14 18:31:47,479 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:47,479 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE"], id: 9988A86CCFC04A42B9BE5DAA07FB70E2
2016-01-14 18:31:47,479 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_REPLACE"]
2016-01-14 18:31:47,479 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:47,480 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:47,480 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:47,480 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 0D55A67ECDEF4C46887240430214B650, for 1 inputs
2016-01-14 18:31:47,480 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:47,480 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : D1F254F4F9E242DCBD6DF8997EE5479C
2016-01-14 18:31:47,480 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:47,480 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: CF832548685B4815BA5DD7D49DDC03B7, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:47,481 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@65605a71; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@18d1e4b6
2016-01-14 18:31:47,481 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:47,483 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:47,484 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:47,484 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:47,485 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796307212_0001_1_01_000000_0
2016-01-14 18:31:47,485 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796307212_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:47,486 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796307212_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:47,488 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:47,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=CF832548685B4815BA5DD7D49DDC03B7, taskAttemptId=attempt_1452796307212_0001_1_01_000000_0, startTime=1452796307456, finishTime=1452796307488, timeTaken=32, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796307480, Process_Duration=5, Process_End_Time=1452796307485, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:47,488 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307212_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:47,488 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796307212_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:47,488 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:47,488 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:TASK_FINISHED]: vertexName=CF832548685B4815BA5DD7D49DDC03B7, taskId=task_1452796307212_0001_1_01_000000, startTime=1452796307456, finishTime=1452796307488, timeTaken=32, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:47,489 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307212_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:47,489 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] : 1
2016-01-14 18:31:47,489 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] because task failed: task_1452796307212_0001_1_01_000000
2016-01-14 18:31:47,489 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:47,489 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:47,489 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:47,489 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7]
2016-01-14 18:31:47,489 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=9988A86CCFC04A42B9BE5DAA07FB70E2, vertexId=vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7]
2016-01-14 18:31:47,490 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:VERTEX_FINISHED]: vertexName=CF832548685B4815BA5DD7D49DDC03B7, vertexId=vertex_1452796307212_0001_1_01, initRequestedTime=1452796307374, initedTime=1452796307379, startRequestedTime=1452796307387, startedTime=1452796307387, finishTime=1452796307490, timeTaken=103, status=FAILED, diagnostics=Task failed, taskId=task_1452796307212_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796307456, firstTasksToStart=[ task_1452796307212_0001_1_01_000000 ], lastTaskFinishTime=1452796307487, lastTasksToFinish=[ task_1452796307212_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:47,491 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:47,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:47,491 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796307212_0001_00_000002, stoppedTime=1452796307491, exitStatus=0
2016-01-14 18:31:47,491 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:47,492 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:47,492 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796307212_0001_1_01_000000_0 failed: true container: container_1452796307212_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:47,492 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:47,492 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:47,492 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796307212_0001_1
2016-01-14 18:31:47,492 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796307212_0001_1_00 [D1F254F4F9E242DCBD6DF8997EE5479C]
2016-01-14 18:31:47,492 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 9988A86CCFC04A42B9BE5DAA07FB70E2 for vertex: vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7]
2016-01-14 18:31:47,493 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796307212_0001_1, queueSize=0
2016-01-14 18:31:47,493 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:47,493 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307212_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796307212_0001_1, startTime=1452796307373, finishTime=1452796307492, timeTaken=119, status=FAILED, diagnostics=Vertex failed, vertexName=CF832548685B4815BA5DD7D49DDC03B7, vertexId=vertex_1452796307212_0001_1_01, diagnostics=[Task failed, taskId=task_1452796307212_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796307212_0001_1_01 [CF832548685B4815BA5DD7D49DDC03B7] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=12, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796307437, Process_Duration=2, Process_End_Time=1452796307439, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=0, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:47,494 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796307212_0001_1 finished with state: FAILED
2016-01-14 18:31:47,494 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796307212_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:47,494 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796307212_0001_00_000002. Relying on regular task shutdown for it to end
2016-01-14 18:31:47 Completed Dag: dag_1452796307212_0001_1
2016-01-14 18:31:47,494 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796307212_0001_1, dagState=FAILED
2016-01-14 18:31:47,494 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796307212_0001_1_post
2016-01-14 18:31:47,494 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:47,494 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:47,494 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000002 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:47,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=591921152, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=5, MERGE_PHASE_TIME=7, FIRST_EVENT_RECEIVED=3, LAST_EVENT_RECEIVED=3][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796307480, Process_Duration=5, Process_End_Time=1452796307485, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:47,495 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: CF832548685B4815BA5DD7D49DDC03B7, mem on close (mb), free: 267, total: 564, max: 672
2016-01-14 18:31:47,495 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=CF832548685B4815BA5DD7D49DDC03B7, index=1
2016-01-14 18:31:47,495 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: D1F254F4F9E242DCBD6DF8997EE5479C
2016-01-14 18:31:47,495 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:47,495 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:47,495 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:47,495 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:47,495 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:47,496 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=CF832548685B4815BA5DD7D49DDC03B7, sourceVertex=D1F254F4F9E242DCBD6DF8997EE5479C
2016-01-14 18:31:47,496 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=CF832548685B4815BA5DD7D49DDC03B7, sourceVertex=9988A86CCFC04A42B9BE5DAA07FB70E2
2016-01-14 18:31:47,496 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796307212_0001_00_000002
2016-01-14 18:31:47,496 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796307212_0001_00_000002
2016-01-14 18:31:47,496 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796307212_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:47,496 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796307212_0001_00_000002 completed but with errors
2016-01-14 18:31:47,497 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796307212_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:47,497 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307212_0001_00_000002 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:47,497 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307212_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:47,506 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...platform/ALL_ARGS_REPLACE, applicationId=application_1452796307212_0001
2016-01-14 18:31:47,507 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:47,508 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:47,508 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:47,508 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:47,508 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:47,508 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:47,508 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:47,508 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:47,508 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:47,508 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:47,509 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:47,509 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:47,509 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:47,512 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/19B9C191FC9B4C88B84577642C021D95/FBBC721BB72F4F969FFB60CD307A5476/.tez/application_1452796307212_0001
2016-01-14 18:31:47,512 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:47,512 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...platform/ALL_ARGS_REPLACE
2016-01-14 18:31:47,512 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:47,513 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.317
2016-01-14 18:31:47,513 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_REPLACE
2016-01-14 18:31:47,534 INFO util.Util (Util.java:findMainClass(1327)) - using default application jar, may cause class not found exceptions on the cluster
2016-01-14 18:31:47,534 INFO planner.Hadoop2TezPlanner (Hadoop2TezPlanner.java:initialize(149)) - using application jar: /opt/concurrent/home/teamcity/.gradle/caches/modules-2/files-2.1/cascading/cascading-hadoop2-tez/3.0.4/561447ee876073bb61fddc52e2ad157f45c8edd/cascading-hadoop2-tez-3.0.4.jar
2016-01-14 18:31:47,564 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: NoHashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.007
2016-01-14 18:31:47,566 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executed rule registry: HashJoinHadoop2TezRuleRegistry, completed as: SUCCESS, in: 00:00.009
2016-01-14 18:31:47,566 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:47,566 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: HashJoinHadoop2TezRuleRegistry, supports assembly with steps: 1, nodes: 2
2016-01-14 18:31:47,566 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] rule registry: NoHashJoinHadoop2TezRuleRegistry, result was selected using: 'default comparator: selects plan with fewest steps and fewest nodes'
2016-01-14 18:31:47,582 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt
2016-01-14 18:31:47,586 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tap forcing step to tez local mode: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP
2016-01-14 18:31:47,627 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting
2016-01-14 18:31:47,628 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:47,628 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] sink: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP"]
2016-01-14 18:31:47,629 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] parallel execution of steps is enabled: false
2016-01-14 18:31:47,629 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] executing total steps: 1
2016-01-14 18:31:47,629 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] allocating management threads: 1
2016-01-14 18:31:47,629 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] starting step: (1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 18:31:47,630 WARN flow.Flow (BaseFlow.java:logWarn(1445)) - [every] 'yarn.timeline-service.enabled' is disabled, please enable to capture detailed metrics of completed flows, this may require starting the YARN timeline server daemon
2016-01-14 18:31:47,641 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] tez session mode enabled: true
2016-01-14 18:31:47,643 INFO app.DAGAppMaster (DAGAppMaster.java:run(731)) - Calling stop for all the services
2016-01-14 18:31:47,643 INFO app.DAGAppMaster (DAGAppMaster.java:run(742)) - Exiting DAGAppMaster..GoodBye!
2016-01-14 18:31:47,645 INFO client.TezClient (TezClient.java:<init>(153)) - Tez Client Version: [ component=tez-api, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:47,645 INFO client.TezClient (TezClient.java:start(333)) - Session mode. Starting session.
2016-01-14 18:31:47,646 INFO client.TezClientUtils (TezClientUtils.java:setupTezJarsLocalResources(159)) - Ignoring 'tez.lib.uris' since 'tez.ignore.lib.uris' is set to true
2016-01-14 18:31:47,654 INFO client.TezClient (TezCommonUtils.java:createTezSystemStagingPath(118)) - Tez system stage directory file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510/.tez/application_1452796307645_0001 doesn't exist and is created
2016-01-14 18:31:47,667 INFO client.LocalClient (LocalClient.java:createDAGAppMaster(328)) - DAGAppMaster thread has been created
2016-01-14 18:31:47,667 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(251)) - DAGAppMaster is not created wait for 100ms...
2016-01-14 18:31:47,668 INFO client.LocalClient (LocalClient.java:run(275)) - Using staging directory: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510/.tez/application_1452796307645_0001
2016-01-14 18:31:47,668 INFO app.DAGAppMaster (DAGAppMaster.java:<init>(309)) - Created DAGAppMaster for application appattempt_1452796307645_0001_000000, versionInfo=[ component=tez-dag, version=0.6.2, revision=c04283bf91fd96823eec37056d5c8b1a92728265, SCM-URL=scm:git:https://git-wip-us.apache.org/repos/asf/tez.git, buildTime=20150804-1436 ]
2016-01-14 18:31:47,678 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(333)) - Comparing client version with AM version, clientVersion=0.6.2, AMVersion=0.6.2
2016-01-14 18:31:47,679 INFO app.DAGAppMaster (DAGAppMaster.java:serviceInit(386)) - Adding session token to jobTokenSecretManager for application
2016-01-14 18:31:47,679 INFO node.AMNodeTracker (AMNodeTracker.java:serviceInit(77)) - blacklistDisablePercent is 33, blacklistingEnabled: false, maxTaskFailuresPerNode: 1
2016-01-14 18:31:47,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceInit(50)) - Initializing HistoryEventHandler
2016-01-14 18:31:47,680 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(66)) - Log file location for SimpleHistoryLoggingService not specified, defaulting to containerLogDir=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510/.tez/application_1452796307645_0001/localmode-log-dir
2016-01-14 18:31:47,680 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceInit(88)) - Initializing SimpleHistoryLoggingService, logFileLocation=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510/.tez/application_1452796307645_0001/localmode-log-dir/history.txt.appattempt_1452796307645_0001_000000, maxErrors=10
2016-01-14 18:31:47,680 INFO recovery.RecoveryService (RecoveryService.java:serviceInit(109)) - Initializing RecoveryService
2016-01-14 18:31:47,680 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: appAttemptId=appattempt_1452796307645_0001_000000, appSubmitTime=1452796307668, launchTime=1452796307668
2016-01-14 18:31:47,684 INFO client.DAGClientServer (DAGClientServer.java:serviceStart(91)) - Instantiated DAGClientRPCServer at ip-10-187-41-40.ec2.internal/10.187.41.40:33932
2016-01-14 18:31:47,685 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStart(73)) - Starting HistoryEventHandler
2016-01-14 18:31:47,685 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStart(95)) - Starting SimpleHistoryLoggingService
2016-01-14 18:31:47,686 INFO recovery.RecoveryService (RecoveryService.java:serviceStart(127)) - Starting RecoveryService
2016-01-14 18:31:47,686 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_LAUNCHED to history file
2016-01-14 18:31:47,687 WARN impl.MetricsConfig (MetricsConfig.java:loadFirst(124)) - Cannot locate configuration: tried hadoop-metrics2-dagappmaster.properties,hadoop-metrics2.properties
2016-01-14 18:31:47,688 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:N/A][Event:AM_STARTED]: appAttemptId=appattempt_1452796307645_0001_000000, startTime=1452796307688
2016-01-14 18:31:47,688 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event AM_STARTED to history file
2016-01-14 18:31:47,688 INFO app.DAGAppMaster (DAGAppMaster.java:serviceStart(1644)) - In Session mode. Waiting for DAG over RPC
2016-01-14 18:31:47,768 INFO client.LocalClient (LocalClient.java:startDAGAppMaster(236)) - DAGAppMaster state: IDLE
2016-01-14 18:31:47,768 INFO client.TezClient (TezClient.java:start(367)) - The url to track the Tez Session: N/A
2016-01-14 18:31:47,768 INFO client.TezClient (TezClient.java:submitDAGSession(405)) - Submitting dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1452796307645_0001, dagName=[396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 18:31:47,785 INFO app.DAGAppMaster (DAGAppMaster.java:submitDAGToAppMaster(1126)) - Starting DAG submitted via RPC: [396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 18:31:47,786 INFO app.DAGAppMaster (DAGAppMaster.java:generateDAGVizFile(870)) - Generating DAG graphviz file, dagId=dag_1452796307645_0001_1, filePath=/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510/.tez/application_1452796307645_0001/localmode-log-dir/dag_1452796307645_0001_1.dot
2016-01-14 18:31:47,786 INFO app.DAGAppMaster (DAGAppMaster.java:writePBTextFile(887)) - Writing DAG plan to: /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510/.tez/application_1452796307645_0001/localmode-log-dir/dag_1452796307645_0001_1-tez-dag.pb.txt
2016-01-14 18:31:47,793 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796307645_0001_1
2016-01-14 18:31:47,793 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:47,793 INFO app.DAGAppMaster (DAGAppMaster.java:startDAG(2051)) - Running DAG: [396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 18:31:47 Running Dag: dag_1452796307645_0001_1
2016-01-14 18:31:47,802 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:DAG_SUBMITTED]: dagID=dag_1452796307645_0001_1, submitTime=1452796307785
2016-01-14 18:31:47,802 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_SUBMITTED to history file
2016-01-14 18:31:47,803 INFO impl.VertexImpl (VertexImpl.java:setAdditionalOutputs(4033)) - setting additional outputs for vertex D8DC787A9EA3491D912A140740897384
2016-01-14 18:31:47,803 INFO impl.DAGImpl (DAGImpl.java:assignDAGScheduler(1381)) - Using DAG Scheduler: org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
2016-01-14 18:31:47,803 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:DAG_INITIALIZED]: dagID=dag_1452796307645_0001_1, initTime=1452796307802
2016-01-14 18:31:47,803 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_INITIALIZED to history file
2016-01-14 18:31:47,804 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796307645_0001_1 transitioned from NEW to INITED due to event DAG_INIT
2016-01-14 18:31:47,804 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:DAG_STARTED]: dagID=dag_1452796307645_0001_1, startTime=1452796307804
2016-01-14 18:31:47,804 INFO impl.DAGImpl (DAGImpl.java:transition(1614)) - Added additional resources : [[]] to classpath
2016-01-14 18:31:47,804 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_STARTED to history file
2016-01-14 18:31:47,804 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796307645_0001_1 transitioned from INITED to RUNNING due to event DAG_START
2016-01-14 18:31:47,804 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2068)) - Root Inputs exist for Vertex: C4C29BDD94924BE781EC5BE50BF8EFCE : {79C84DF8B9E44BFDA2D01549A92AB235={InputName=79C84DF8B9E44BFDA2D01549A92AB235}, {Descriptor=ClassName=org.apache.tez.mapreduce.input.MRInput, hasPayload=true}, {ControllerDescriptor=ClassName=org.apache.tez.mapreduce.common.MRInputAMSplitGenerator, hasPayload=false}}
2016-01-14 18:31:47,805 INFO impl.VertexImpl (VertexImpl.java:setupVertex(2078)) - Starting root input initializer for input: 79C84DF8B9E44BFDA2D01549A92AB235, with class: [org.apache.tez.mapreduce.common.MRInputAMSplitGenerator]
2016-01-14 18:31:47,805 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2191)) - Setting vertexManager to RootInputVertexManager for vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE]
2016-01-14 18:31:47,805 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2983)) - Num tasks is -1. Expecting VertexManager/InputInitializers/1-1 split to set #tasks for the vertex vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE]
2016-01-14 18:31:47,805 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(2987)) - Vertex will initialize from input initializer. vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE]
2016-01-14 18:31:47,805 INFO impl.VertexImpl (VertexImpl.java:setupInputInitializerManager(3933)) - Vertex will initialize via inputInitializers vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE]. Starting root input initializers: 1
2016-01-14 18:31:47,805 INFO client.TezClient (TezClient.java:submitDAGSession(478)) - Submitted dag to TezSession, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1452796307645_0001, dagName=[396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510] every/(1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 18:31:47,805 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE] transitioned from NEW to INITIALIZING due to event V_INIT
2016-01-14 18:31:47,806 INFO impl.VertexImpl (VertexImpl.java:assignVertexManager(2203)) - Setting vertexManager to ShuffleVertexManager for vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384]
2016-01-14 18:31:47,807 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:run(242)) - Starting InputInitializer for Input: 79C84DF8B9E44BFDA2D01549A92AB235 on vertex vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE]
2016-01-14 18:31:47,809 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:initialize(696)) - Shuffle Vertex Manager: settings minFrac:0.25 maxFrac:0.75 auto:false desiredTaskIput:104857600 minTasks:1
2016-01-14 18:31:47,809 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3014)) - Creating 1 tasks for vertex: vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384]
2016-01-14 18:31:47,809 INFO impl.VertexImpl (VertexImpl.java:handleInitEvent(3026)) - Directly initializing vertex: vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384]
2016-01-14 18:31:47,809 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1915)) - Invoking committer inits for vertex, vertexId=vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384]
2016-01-14 18:31:47,809 INFO impl.VertexImpl (VertexImpl.java:initializeCommitters(1927)) - Instantiating committer for output=3831E03A344D48F6A1078CE41C53AD00, vertexId=vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384], committerClass=org.apache.tez.mapreduce.committer.MROutputCommitter
2016-01-14 18:31:47,810 INFO impl.VertexImpl (VertexImpl.java:run(1945)) - Invoking committer init for output=3831E03A344D48F6A1078CE41C53AD00, vertexId=vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384]
2016-01-14 18:31:47,810 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(92)) - Emitting serialized splits: true
2016-01-14 18:31:47,810 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(110)) - Input 79C84DF8B9E44BFDA2D01549A92AB235 asking for 1 tasks. Headroom: 672 Task Resource: 1024 waves: 1.7
2016-01-14 18:31:47,811 INFO common.MRInputAMSplitGenerator (MRInputAMSplitGenerator.java:initialize(121)) - Grouping input splits
2016-01-14 18:31:47,811 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(294)) - Generating mapred api input splits
2016-01-14 18:31:47,812 INFO committer.MROutputCommitter (MROutputCommitter.java:getOutputCommitter(145)) - OutputCommitter for outputName=3831E03A344D48F6A1078CE41C53AD00, vertexName=D8DC787A9EA3491D912A140740897384, outputCommitterClass=org.apache.hadoop.mapred.FileOutputCommitter
2016-01-14 18:31:47,812 INFO impl.VertexImpl (VertexImpl.java:run(1949)) - Invoking committer setup for output=3831E03A344D48F6A1078CE41C53AD00, vertexId=vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384]
2016-01-14 18:31:47,813 INFO hadoop.MRInputHelpers (MRInputHelpers.java:generateInputSplitsToMem(300)) - NumSplits: 1, SerializedSize: 404
2016-01-14 18:31:47,813 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:47,813 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:VERTEX_INITIALIZED]: vertexName=D8DC787A9EA3491D912A140740897384, vertexId=vertex_1452796307645_0001_1_01, initRequestedTime=1452796307806, initedTime=1452796307813, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=0
2016-01-14 18:31:47,813 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] transitioned from NEW to INITED due to event V_INIT
2016-01-14 18:31:47,813 INFO dag.RootInputInitializerManager (RootInputInitializerManager.java:onSuccess(272)) - Succeeded InputInitializer for Input: 79C84DF8B9E44BFDA2D01549A92AB235 on vertex vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE]
2016-01-14 18:31:47,814 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE] numTasks: -1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:47,814 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1374)) - Got updated RootInputsSpecs: {79C84DF8B9E44BFDA2D01549A92AB235=forAllWorkUnits=true, update=[1]}
2016-01-14 18:31:47,814 INFO impl.VertexImpl (VertexImpl.java:setParallelism(1393)) - Vertex vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE] parallelism set to 1
2016-01-14 18:31:47,814 INFO impl.VertexImpl (VertexImpl.java:canInitVertex(3195)) - Cannot init vertex: vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE] numTasks: 1 numUnitializedEdges: 0 numInitializedInputs: 0 initWaitsForRootInitializers: true
2016-01-14 18:31:47,814 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796307645_0001_1_00, eventCount=1
2016-01-14 18:31:47,815 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_INITIALIZED to history file
2016-01-14 18:31:47,815 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:VERTEX_INITIALIZED]: vertexName=C4C29BDD94924BE781EC5BE50BF8EFCE, vertexId=vertex_1452796307645_0001_1_00, initRequestedTime=1452796307804, initedTime=1452796307815, numTasks=1, processorName=cascading.flow.tez.FlowProcessor, additionalInputsCount=1
2016-01-14 18:31:47,815 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE] with distanceFromRoot: 0
2016-01-14 18:31:47,815 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE] transitioned from INITIALIZING to INITED due to event V_ROOT_INPUT_INITIALIZED
2016-01-14 18:31:47,816 INFO impl.ImmediateStartVertexManager (ImmediateStartVertexManager.java:scheduleTasks(99)) - Starting 1 in C4C29BDD94924BE781EC5BE50BF8EFCE
2016-01-14 18:31:47,816 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE]
2016-01-14 18:31:47,816 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:47,816 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796307645_0001_1_00, startRequestedTime=1452796307815, startedTime=1452796307815
2016-01-14 18:31:47,816 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:47,817 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:47,817 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:TASK_STARTED]: vertexName=C4C29BDD94924BE781EC5BE50BF8EFCE, taskId=task_1452796307645_0001_1_00_000000, scheduledTime=1452796307817, launchTime=1452796307817
2016-01-14 18:31:47,817 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307645_0001_1_00_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:47,818 INFO impl.VertexImpl (VertexImpl.java:transition(3155)) - Source vertex started: vertex_1452796307645_0001_1_00 for vertex: vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] numStartedSources: 1 numSources: 1
2016-01-14 18:31:47,818 INFO impl.VertexImpl (VertexImpl.java:startIfPossible(3061)) - Triggering start event for vertex: vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] with distanceFromRoot: 1
2016-01-14 18:31:47,818 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE]
2016-01-14 18:31:47,818 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:handleVertexStateUpdate(720)) - Received configured notification : CONFIGURED for vertex: C4C29BDD94924BE781EC5BE50BF8EFCE in vertex: D8DC787A9EA3491D912A140740897384
2016-01-14 18:31:47,819 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:onVertexStarted(355)) - OnVertexStarted vertex: D8DC787A9EA3491D912A140740897384 with 1 source tasks and 1 pending tasks
2016-01-14 18:31:47,819 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_STARTED to history file
2016-01-14 18:31:47,819 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:VERTEX_STARTED]: vertexId=vertex_1452796307645_0001_1_01, startRequestedTime=1452796307818, startedTime=1452796307818
2016-01-14 18:31:47,819 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] transitioned from INITED to RUNNING due to event V_START
2016-01-14 18:31:47,820 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: C4C29BDD94924BE781EC5BE50BF8EFCE, VertexParallelism: 1, TaskAttemptID:attempt_1452796307645_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=79C84DF8B9E44BFDA2D01549A92AB235, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D8DC787A9EA3491D912A140740897384, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:47,820 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307645_0001_1_00_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:47,820 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:47,821 INFO node.AMNodeTracker (AMNodeTracker.java:nodeSeen(91)) - Adding new node: 127.0.0.1:0
2016-01-14 18:31:47,821 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(112)) - Adding #1 tokens and #0 secret keys for NM use for launching container
2016-01-14 18:31:47,821 INFO container.AMContainerHelpers (AMContainerHelpers.java:createCommonContainerLaunchContext(122)) - Putting shuffle token in serviceData
2016-01-14 18:31:47,822 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796307645_0001_00_000001
2016-01-14 18:31:47,822 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000001 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:47,823 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796307645_0001_00_000001
2016-01-14 18:31:47,823 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:47,824 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000001 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:47,824 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:47,824 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796307645_0001_00_000001, launchTime=1452796307824
2016-01-14 18:31:47,825 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796307645_0001_00_000001
2016-01-14 18:31:47,825 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796307645_0001_1_00_000000_0] to container: [container_1452796307645_0001_00_000001]
2016-01-14 18:31:47,826 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000001 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:47,826 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796307645_0001_1_00_000000_0] started. Is using containerId: [container_1452796307645_0001_00_000001] on NM: [127.0.0.1:0]
2016-01-14 18:31:47,826 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=C4C29BDD94924BE781EC5BE50BF8EFCE, taskAttemptId=attempt_1452796307645_0001_1_00_000000_0, startTime=1452796307826, containerId=container_1452796307645_0001_00_000001, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796307645_0001_00_000001/teamcity, completedLogs=
2016-01-14 18:31:47,826 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307645_0001_1_00_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:47,827 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307645_0001_1_00_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:47,827 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:47,827 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796307645_0001_00_000001 given task: attempt_1452796307645_0001_1_00_000000_0
2016-01-14 18:31:47,827 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796307645_0001_00_000001: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796307645_0001_1_00_000000_0
2016-01-14 18:31:47 Starting to run new task attempt: attempt_1452796307645_0001_1_00_000000_0
2016-01-14 18:31:47,828 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796307645_0001_1_00_000000_0
2016-01-14 18:31:47,828 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:47,828 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:47,828 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:47,828 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:47,828 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:47,828 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:47,828 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: C4C29BDD94924BE781EC5BE50BF8EFCE, VertexParallelism: 1, TaskAttemptID:attempt_1452796307645_0001_1_00_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=79C84DF8B9E44BFDA2D01549A92AB235, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }}, ], outputSpecList=[{{ destinationVertexName=D8DC787A9EA3491D912A140740897384, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }}, ]
2016-01-14 18:31:47,829 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:47,829 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796307645_0001_1_00_000000_0
2016-01-14 18:31:47,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:47,829 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,829 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796307645_0001_1_00_000000_0 sent events: (0-1)
2016-01-14 18:31:47,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=79C84DF8B9E44BFDA2D01549A92AB235, physicalEdgeCount=1, inputClassName=org.apache.tez.mapreduce.input.MRInput }
2016-01-14 18:31:47,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:47,832 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: 79C84DF8B9E44BFDA2D01549A92AB235
2016-01-14 18:31:47,832 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: INPUT, componentVertexName: 79C84DF8B9E44BFDA2D01549A92AB235
2016-01-14 18:31:47,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=D8DC787A9EA3491D912A140740897384, physicalEdgeCount=1, outputClassName=org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput }
2016-01-14 18:31:47,833 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:47,834 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: D8DC787A9EA3491D912A140740897384
2016-01-14 18:31:47,835 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] submitted tez dag to app master: application_1452796307645_0001, with dag id: dag_1452796307645_0001_1
2016-01-14 18:31:47,835 INFO impl.ExternalSorter (ExternalSorter.java:getInitialMemoryRequirement(293)) - Requested SortBufferSize (tez.runtime.io.sort.mb): 100
2016-01-14 18:31:47,836 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 104857600, type: OUTPUT, componentVertexName: D8DC787A9EA3491D912A140740897384
2016-01-14 18:31:47,836 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: D8DC787A9EA3491D912A140740897384
2016-01-14 18:31:47,836 INFO input.MRInput (MRInput.java:initialize(440)) - Using New mapreduce API: false, split information via event: true
2016-01-14 18:31:47,836 INFO input.MRInput (MRInput.java:initializeInternal(493)) - Initialzed MRInput: 79C84DF8B9E44BFDA2D01549A92AB235
2016-01-14 18:31:47,837 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: 79C84DF8B9E44BFDA2D01549A92AB235
2016-01-14 18:31:47,838 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:47,841 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: C4C29BDD94924BE781EC5BE50BF8EFCE, ordinal: 0
2016-01-14 18:31:47,841 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C4C29BDD94924BE781EC5BE50BF8EFCE, mem on start (mb), free: 180, total: 564, max: 672
2016-01-14 18:31:47,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:47,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:47,842 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:47,842 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:47,842 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.input.MRInput
2016-01-14 18:31:47,842 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:47,842 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_OUTPUT
2016-01-14 18:31:47,842 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:47,842 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 104857600, TotalRequestedScaled: 9.679163076923077E7, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.15
2016-01-14 18:31:47,843 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, 79C84DF8B9E44BFDA2D01549A92AB235, org.apache.tez.mapreduce.input.MRInput: requested=0, allocated=0
2016-01-14 18:31:47,843 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, D8DC787A9EA3491D912A140740897384, org.apache.tez.runtime.library.output.OrderedPartitionedKVOutput: requested=104857600, allocated=104857600
2016-01-14 18:31:47,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:47,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: 79C84DF8B9E44BFDA2D01549A92AB235 being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:47,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:47,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:47,843 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: 79C84DF8B9E44BFDA2D01549A92AB235
2016-01-14 18:31:47,844 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:47,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: 79C84DF8B9E44BFDA2D01549A92AB235
2016-01-14 18:31:47,844 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:47,844 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796307645_0001_1_00_000000_0
2016-01-14 18:31:47,844 INFO input.MRInput (MRInput.java:initFromEventInternal(619)) - Initializing RecordReader from event
2016-01-14 18:31:47,845 INFO input.MRInput (MRInput.java:initFromEventInternal(630)) - Split Details -> SplitClass: org.apache.hadoop.mapred.split.TezGroupedSplit, OldSplit: org.apache.hadoop.mapred.split.TezGroupedSplit@42a29561
2016-01-14 18:31:47,845 INFO lib.MRReaderMapred (MRReaderMapred.java:setIncrementalConfigParams(163)) - Processing split: org.apache.hadoop.mapred.split.TezGroupedSplit@42a29561
2016-01-14 18:31:47,846 INFO input.MRInput (MRInput.java:initFromEventInternal(634)) - Initialized RecordReader from event
2016-01-14 18:31:47,846 INFO input.MRInput (MRInput.java:processSplitEvent(586)) - Notifying on RecordReader Initialized
2016-01-14 18:31:47,847 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:47,847 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"] streamed: true, id: 79C84DF8B9E44BFDA2D01549A92AB235
2016-01-14 18:31:47,847 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: GroupBy(every)[by:[{?}:ALL]], id: 9BF40797CC364FF488F6D0231D0C6313
2016-01-14 18:31:47,848 INFO element.TezGroupGate (TezGroupGate.java:prepare(96)) - calling OrderedPartitionedKVOutput#start() on: GroupBy(every)[by:[{?}:ALL]] 9BF40797CC364FF488F6D0231D0C6313
2016-01-14 18:31:47,848 INFO impl.ExternalSorter (ExternalSorter.java:<init>(167)) - keySerializer=cascading.tuple.hadoop.io.TupleSerializer@379217ae; valueSerializer=cascading.tuple.hadoop.io.TupleSerializer@5850bd02; comparator=cascading.tuple.hadoop.util.TupleComparator@1ca8fb9a; conf=cascading.tuple.hadoop.TupleSerialization,org.apache.hadoop.io.serializer.WritableSerialization,org.apache.hadoop.io.serializer.avro.AvroSpecificSerialization,org.apache.hadoop.io.serializer.avro.AvroReflectSerialization
2016-01-14 18:31:47,848 INFO impl.ExternalSorter (ExternalSorter.java:<init>(224)) - Instantiating Partitioner: [cascading.tuple.tez.util.TuplePartitioner]
2016-01-14 18:31:47,848 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiatePartitioner(109)) - Using partitioner class: cascading.tuple.tez.util.TuplePartitioner
2016-01-14 18:31:47,848 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:47,870 INFO dflt.DefaultSorter (DefaultSorter.java:setEquator(328)) - (EQUATOR) 0 kvi 26214396(104857584)
2016-01-14 18:31:47,870 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(150)) - tez.runtime.io.sort.mb: 100
2016-01-14 18:31:47,870 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(151)) - soft limit at 83886080
2016-01-14 18:31:47,871 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(152)) - bufstart = 0; bufvoid = 104857600
2016-01-14 18:31:47,871 INFO dflt.DefaultSorter (DefaultSorter.java:<init>(153)) - kvstart = 26214396; length = 6553600
2016-01-14 18:31:47,872 INFO element.TezSourceStage (TezSourceStage.java:prepare(63)) - calling MRInput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/classes/test/data/nums.20.txt"]
2016-01-14 18:31:47,873 INFO input.MRInput (MRInput.java:start(450)) - MRInput setup to received events1
2016-01-14 18:31:47,873 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: C4C29BDD94924BE781EC5BE50BF8EFCE, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:47,875 INFO task.TezTaskRunner (TezTaskRunner.java:run(177)) - Closing task, taskAttemptId=attempt_1452796307645_0001_1_00_000000_0
2016-01-14 18:31:47,875 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: C4C29BDD94924BE781EC5BE50BF8EFCE, mem on close (mb), free: 275, total: 565, max: 672
2016-01-14 18:31:47,875 INFO dflt.DefaultSorter (DefaultSorter.java:flush(589)) - Starting flush of map output
2016-01-14 18:31:47,875 INFO dflt.DefaultSorter (DefaultSorter.java:flush(607)) - Sorting & Spilling map output
2016-01-14 18:31:47,875 INFO dflt.DefaultSorter (DefaultSorter.java:flush(608)) - bufstart = 0; bufend = 251; bufvoid = 104857600
2016-01-14 18:31:47,875 INFO dflt.DefaultSorter (DefaultSorter.java:flush(610)) - kvstart = 26214396(104857584); kvend = 26214320(104857280); length = 77/6553600
2016-01-14 18:31:47,877 INFO dflt.DefaultSorter (DefaultSorter.java:spill(829)) - Finished spill 0
2016-01-14 18:31:47 Completed running task attempt: attempt_1452796307645_0001_1_00_000000_0
2016-01-14 18:31:47,879 INFO task.TezTaskRunner (TezTaskRunner.java:run(181)) - Task completed, taskAttemptId=attempt_1452796307645_0001_1_00_000000_0, fatalErrorOccurred=false
2016-01-14 18:31:47,880 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 27 [[File System Counters FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0][cascading.flow.SliceCounters Process_Begin_Time=1452796307873, Process_Duration=2, Process_End_Time=1452796307875, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2][cascading.flow.StepCounters Tuples_Read=20]]
2016-01-14 18:31:47,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:VERTEX_DATA_MOVEMENT_EVENTS_GENERATED]: vertexId=vertex_1452796307645_0001_1_00, eventCount=1
2016-01-14 18:31:47,880 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend:
2016-01-14 18:31:47,880 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:47,880 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796307645_0001_00_000001
2016-01-14 18:31:47,880 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=C4C29BDD94924BE781EC5BE50BF8EFCE, taskAttemptId=attempt_1452796307645_0001_1_00_000000_0, startTime=1452796307826, finishTime=1452796307880, timeTaken=54, status=SUCCEEDED, errorEnum=, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796307873, Process_Duration=2, Process_End_Time=1452796307875, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:47,880 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:47,880 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307645_0001_1_00_000000_0 TaskAttempt Transitioned from RUNNING to SUCCEEDED due to event TA_DONE
2016-01-14 18:31:47,881 INFO impl.TaskImpl (TaskImpl.java:transition(1052)) - Task succeeded with attempt attempt_1452796307645_0001_1_00_000000_0
2016-01-14 18:31:47,881 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:47,881 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:47,881 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:TASK_FINISHED]: vertexName=C4C29BDD94924BE781EC5BE50BF8EFCE, taskId=task_1452796307645_0001_1_00_000000, startTime=1452796307826, finishTime=1452796307881, timeTaken=55, status=SUCCEEDED, successfulAttemptID=attempt_1452796307645_0001_1_00_000000_0, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796307873, Process_Duration=2, Process_End_Time=1452796307875, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:47,881 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307645_0001_1_00_000000 Task Transitioned from RUNNING to SUCCEEDED due to event T_ATTEMPT_SUCCEEDED
2016-01-14 18:31:47,881 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE] : 1
2016-01-14 18:31:47,881 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1674)) - Vertex succeeded: vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE]
2016-01-14 18:31:47,882 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:47,882 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:VERTEX_FINISHED]: vertexName=C4C29BDD94924BE781EC5BE50BF8EFCE, vertexId=vertex_1452796307645_0001_1_00, initRequestedTime=1452796307804, initedTime=1452796307815, startRequestedTime=1452796307815, startedTime=1452796307815, finishTime=1452796307881, timeTaken=66, status=SUCCEEDED, diagnostics=, counters=Counters: 28, org.apache.tez.common.counters.DAGCounter, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796307873, Process_Duration=2, Process_End_Time=1452796307875, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20, vertexStats=firstTaskStartTime=1452796307826, firstTasksToStart=[ task_1452796307645_0001_1_00_000000 ], lastTaskFinishTime=1452796307880, lastTasksToFinish=[ task_1452796307645_0001_1_00_000000 ], minTaskDuration=54, maxTaskDuration=54, avgTaskDuration=54.0, numSuccessfulTasks=1, shortestDurationTasks=[ task_1452796307645_0001_1_00_000000 ], longestDurationTasks=[ task_1452796307645_0001_1_00_000000 ], vertexTaskStats={numFailedTasks=0, numSucceededTasks=1, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=0, numCompletedTasks=1}
2016-01-14 18:31:47,882 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE] transitioned from RUNNING to SUCCEEDED due to event V_TASK_COMPLETED
2016-01-14 18:31:47,882 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000001 transitioned from RUNNING to IDLE via event C_TA_SUCCEEDED
2016-01-14 18:31:47,882 INFO impl.VertexImpl (VertexImpl.java:transition(3477)) - Source task attempt completed for vertex: vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] attempt: attempt_1452796307645_0001_1_00_000000_0 with state: SUCCEEDED vertexState: RUNNING
2016-01-14 18:31:47,882 INFO vertexmanager.ShuffleVertexManager (ShuffleVertexManager.java:schedulePendingTasks(608)) - All source tasks assigned. Ramping up 1 remaining tasks for vertex: D8DC787A9EA3491D912A140740897384
2016-01-14 18:31:47,882 INFO impl.VertexImpl (VertexImpl.java:scheduleTasks(1250)) - Routing pending task events for vertex: vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384]
2016-01-14 18:31:47,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796307645_0001_00_000001, stoppedTime=1452796307882, exitStatus=0
2016-01-14 18:31:47,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:47,883 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000001 transitioned from IDLE to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:47,883 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE] completed., numCompletedVertices=1, numSuccessfulVertices=1, numFailedVertices=0, numKilledVertices=0, numVertices=2
2016-01-14 18:31:47,883 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:TASK_STARTED]: vertexName=D8DC787A9EA3491D912A140740897384, taskId=task_1452796307645_0001_1_01_000000, scheduledTime=1452796307883, launchTime=1452796307883
2016-01-14 18:31:47,883 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_STARTED to history file
2016-01-14 18:31:47,884 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307645_0001_1_01_000000 Task Transitioned from NEW to SCHEDULED due to event T_SCHEDULE
2016-01-14 18:31:47,884 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(244)) - Ignoring stop request for containerId container_1452796307645_0001_00_000001. Relying on regular task shutdown for it to end
2016-01-14 18:31:47,884 INFO impl.DAGSchedulerNaturalOrderControlled (DAGSchedulerNaturalOrderControlled.java:scheduleTask(111)) - Scheduled vertex: vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384]
2016-01-14 18:31:47,885 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000001 transitioned from STOP_REQUESTED to STOPPING via event C_NM_STOP_SENT
2016-01-14 18:31:47,885 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1014)) - remoteTaskSpec:DAGName : [396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: D8DC787A9EA3491D912A140740897384, VertexParallelism: 1, TaskAttemptID:attempt_1452796307645_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C4C29BDD94924BE781EC5BE50BF8EFCE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3831E03A344D48F6A1078CE41C53AD00, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:47,885 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307645_0001_1_01_000000_0 TaskAttempt Transitioned from NEW to START_WAIT due to event TA_SCHEDULE
2016-01-14 18:31:47,885 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_LAUNCH_REQUEST
2016-01-14 18:31:47,886 INFO container.AMContainerImpl (AMContainerImpl.java:transition(521)) - Sending Launch Request for Container with id: container_1452796307645_0001_00_000002
2016-01-14 18:31:47,886 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000002 transitioned from ALLOCATED to LAUNCHING via event C_LAUNCH_REQUEST
2016-01-14 18:31:47,887 INFO task.TezChild (TezChild.java:newTezChild(432)) - PID, containerIdentifier: null, container_1452796307645_0001_00_000002
2016-01-14 18:31:47,887 WARN impl.MetricsSystemImpl (MetricsSystemImpl.java:init(151)) - DAGAppMaster metrics system already initialized!
2016-01-14 18:31:47,887 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000002 transitioned from LAUNCHING to IDLE via event C_LAUNCHED
2016-01-14 18:31:47,888 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_LAUNCHED to history file
2016-01-14 18:31:47,888 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:CONTAINER_LAUNCHED]: containerId=container_1452796307645_0001_00_000002, launchTime=1452796307887
2016-01-14 18:31:47,888 INFO task.ContainerReporter (ContainerReporter.java:call(55)) - Attempting to fetch new task for container container_1452796307645_0001_00_000002
2016-01-14 18:31:47,889 INFO container.AMContainerImpl (AMContainerImpl.java:transition(827)) - Assigned taskAttempt + [attempt_1452796307645_0001_1_01_000000_0] to container: [container_1452796307645_0001_00_000002]
2016-01-14 18:31:47,889 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000002 transitioned from IDLE to RUNNING via event C_PULL_TA
2016-01-14 18:31:47,889 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(240)) - Container with id: container_1452796307645_0001_00_000002 given task: attempt_1452796307645_0001_1_01_000000_0
2016-01-14 18:31:47,889 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796307645_0001_00_000002: 0 ms after starting to poll. TaskInfo: shouldDie: false, currentTaskAttemptId: attempt_1452796307645_0001_1_01_000000_0
2016-01-14 18:31:47,889 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:transition(1146)) - TaskAttempt: [attempt_1452796307645_0001_1_01_000000_0] started. Is using containerId: [container_1452796307645_0001_00_000002] on NM: [127.0.0.1:0]
2016-01-14 18:31:47,890 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:TASK_ATTEMPT_STARTED]: vertexName=D8DC787A9EA3491D912A140740897384, taskAttemptId=attempt_1452796307645_0001_1_01_000000_0, startTime=1452796307889, containerId=container_1452796307645_0001_00_000002, nodeId=127.0.0.1:0, inProgressLogs=127.0.0.1:0/node/containerlogs/container_1452796307645_0001_00_000002/teamcity, completedLogs=
2016-01-14 18:31:47,890 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307645_0001_1_01_000000_0 TaskAttempt Transitioned from START_WAIT to RUNNING due to event TA_STARTED_REMOTELY
2016-01-14 18:31:47,890 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_STARTED to history file
2016-01-14 18:31:47,890 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307645_0001_1_01_000000 Task Transitioned from SCHEDULED to RUNNING due to event T_ATTEMPT_LAUNCHED
2016-01-14 18:31:47,891 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:getTask(213)) - Container with id: container_1452796307645_0001_00_000001 is valid, but no longer registered, and will be killed
2016-01-14 18:31:47,891 INFO task.ContainerReporter (ContainerReporter.java:call(65)) - Got TaskUpdate for containerId= container_1452796307645_0001_00_000001: 11 ms after starting to poll. TaskInfo: shouldDie: true
2016-01-14 18:31:47,891 INFO task.TezChild (TezChild.java:run(207)) - ContainerTask returned shouldDie=true for container container_1452796307645_0001_00_000001, Exiting
2016-01-14 18:31:47,891 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796307645_0001_00_000001
2016-01-14 18:31:47,892 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796307645_0001_00_000001
2016-01-14 18:31:47 Starting to run new task attempt: attempt_1452796307645_0001_1_01_000000_0
2016-01-14 18:31:47,892 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: attempt_1452796307645_0001_1_01_000000_0
2016-01-14 18:31:47,892 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796307645_0001_00_000001 = ContainerExecutionResult{exitStatus=SUCCESS, throwable=null, errorMessage='Asked to die by the AM'}
2016-01-14 18:31:47,893 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(274)) - Container: container_1452796307645_0001_00_000001 completed successfully
2016-01-14 18:31:47,893 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:47,893 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(273)) - Refreshing UGI since Credentials have changed
2016-01-14 18:31:47,893 INFO task.TezChild (TezChild.java:handleNewTaskCredentials(276)) - Credentials : #Tokens=1, #SecretKeys=0
2016-01-14 18:31:47,893 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(302)) - Localizing additional local resources for Task : {}
2016-01-14 18:31:47,893 INFO task.TezChild (TezChild.java:handleNewTaskLocalResources(312)) - Done localizing additional resources
2016-01-14 18:31:47,893 INFO metrics.TaskCounterUpdater (TaskCounterUpdater.java:initResourceCalculatorPlugin(148)) - Using ResourceCalculatorProcessTree : [ ]
2016-01-14 18:31:47,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:<init>(149)) - Initializing LogicalIOProcessorRuntimeTask with TaskSpec: DAGName : [396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510] every/(1/1) ...blyplatform/ALL_ARGS_SWAP, VertexName: D8DC787A9EA3491D912A140740897384, VertexParallelism: 1, TaskAttemptID:attempt_1452796307645_0001_1_01_000000_0, processorName=cascading.flow.tez.FlowProcessor, inputSpecListSize=1, outputSpecListSize=1, inputSpecList=[{{ sourceVertexName=C4C29BDD94924BE781EC5BE50BF8EFCE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }}, ], outputSpecList=[{{ destinationVertexName=3831E03A344D48F6A1078CE41C53AD00, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }}, ]
2016-01-14 18:31:47,894 INFO resources.MemoryDistributor (MemoryDistributor.java:<init>(85)) - InitialMemoryDistributor (isEnabled=true) invoked with: numInputs=1, numOutputs=1, JVM.maxFree=704643072
2016-01-14 18:31:47,894 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000001 transitioned from STOPPING to COMPLETED via event C_COMPLETED
2016-01-14 18:31:47,894 INFO task.TezTaskRunner (TezTaskRunner.java:run(172)) - Initializing task, taskAttemptId=attempt_1452796307645_0001_1_01_000000_0
2016-01-14 18:31:47,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(191)) - Initializing LogicalProcessorIORuntimeTask
2016-01-14 18:31:47,894 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(195)) - Creating processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(488)) - Initializing processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(389)) - Initializing Input using InputSpec: { sourceVertexName=C4C29BDD94924BE781EC5BE50BF8EFCE, physicalEdgeCount=1, inputClassName=org.apache.tez.runtime.library.input.OrderedGroupedKVInput }
2016-01-14 18:31:47,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createInput(537)) - Creating Input
2016-01-14 18:31:47,895 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(397)) - Initializing Input with src edge: C4C29BDD94924BE781EC5BE50BF8EFCE
2016-01-14 18:31:47,896 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(438)) - Initializing Output using OutputSpec: { destinationVertexName=3831E03A344D48F6A1078CE41C53AD00, physicalEdgeCount=0, outputClassName=org.apache.tez.mapreduce.output.MROutput }
2016-01-14 18:31:47,897 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(310)) - Initial Shuffle Memory Required: 634178752, based on INPUT_BUFFER_factor: 0.9
2016-01-14 18:31:47,897 INFO orderedgrouped.MergeManager (MergeManager.java:getInitialMemoryRequirement(320)) - Initial Memory required for final merged output: 0, using factor: 0.0
2016-01-14 18:31:47,898 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 634178752, type: INPUT, componentVertexName: C4C29BDD94924BE781EC5BE50BF8EFCE
2016-01-14 18:31:47,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(403)) - Initialized Input with src edge: C4C29BDD94924BE781EC5BE50BF8EFCE
2016-01-14 18:31:47,898 INFO tez.FlowProcessor (FlowProcessor.java:initialize(88)) - cascading version: 3.0.4
2016-01-14 18:31:47,898 INFO impl.TaskImpl (TaskImpl.java:getTaskAttemptTezEvents(481)) - TaskAttempt:attempt_1452796307645_0001_1_01_000000_0 sent events: (0-1)
2016-01-14 18:31:47,898 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:createOutput(565)) - Creating Output
2016-01-14 18:31:47,899 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(446)) - Initializing Output with dest edge: 3831E03A344D48F6A1078CE41C53AD00
2016-01-14 18:31:47,899 INFO output.MROutput (MROutput.java:initialize(347)) - Initializing Simple Output
2016-01-14 18:31:47,899 INFO resources.MemoryDistributor$RequestorInfo (MemoryDistributor.java:<init>(233)) - Received request: 0, type: OUTPUT, componentVertexName: 3831E03A344D48F6A1078CE41C53AD00
2016-01-14 18:31:47,902 INFO tez.FlowProcessor (FlowProcessor.java:initialize(94)) - flow node id: D8DC787A9EA3491D912A140740897384, ordinal: 1
2016-01-14 18:31:47,902 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8DC787A9EA3491D912A140740897384, mem on start (mb), free: 269, total: 565, max: 672
2016-01-14 18:31:47,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initializeLogicalIOProcessor(491)) - Initialized processor, processorClassName=cascading.flow.tez.FlowProcessor
2016-01-14 18:31:47,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 2 initializers to finish
2016-01-14 18:31:47,902 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(220)) - Waiting for 1 initializers to finish
2016-01-14 18:31:47,908 INFO output.MROutput (MROutput.java:initialize(418)) - Initialized Simple Output, using_new_api: false
2016-01-14 18:31:47,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(452)) - Initialized Output with dest edge: 3831E03A344D48F6A1078CE41C53AD00
2016-01-14 18:31:47,908 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(233)) - All initializers finished
2016-01-14 18:31:47,908 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(124)) - Using Allocator class: org.apache.tez.runtime.library.resources.WeightedScalingMemoryDistributor
2016-01-14 18:31:47,908 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 12, for type: SORTED_MERGED_INPUT
2016-01-14 18:31:47,909 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:getRequestTypeForClass(197)) - Falling back to RequestType.OTHER for class: org.apache.tez.mapreduce.output.MROutput
2016-01-14 18:31:47,909 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:initialProcessMemoryRequestContext(171)) - ScaleFactor: 1, for type: OTHER
2016-01-14 18:31:47,909 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:computeReservedFraction(262)) - InitialReservationFraction=0.3, AdditionalReservationFractionForIOs=0.03, finalReserveFractionUsed=0.32999999999999996
2016-01-14 18:31:47,909 INFO resources.WeightedScalingMemoryDistributor (WeightedScalingMemoryDistributor.java:assignMemory(126)) - Scaling Requests. NumRequests: 2, numScaledRequests: 13, TotalRequested: 634178752, TotalRequestedScaled: 5.853957710769231E8, TotalJVMHeap: 704643072, TotalAvailable: 472110858, TotalRequested/TotalJVMHeap:0.90
2016-01-14 18:31:47,909 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: INPUT, C4C29BDD94924BE781EC5BE50BF8EFCE, org.apache.tez.runtime.library.input.OrderedGroupedKVInput: requested=634178752, allocated=472110858
2016-01-14 18:31:47,910 INFO resources.MemoryDistributor (MemoryDistributor.java:makeInitialAllocations(138)) - Informing: OUTPUT, 3831E03A344D48F6A1078CE41C53AD00, org.apache.tez.mapreduce.output.MROutput: requested=0, allocated=0
2016-01-14 18:31:47,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(255)) - Starting Inputs/Outputs
2016-01-14 18:31:47,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(268)) - Input: C4C29BDD94924BE781EC5BE50BF8EFCE being auto started by the framework. Subsequent instances will not be auto-started
2016-01-14 18:31:47,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(419)) - Starting Input with src edge: C4C29BDD94924BE781EC5BE50BF8EFCE
2016-01-14 18:31:47,910 INFO common.TezRuntimeUtils (TezRuntimeUtils.java:instantiateCombiner(65)) - No combiner specified via tez.runtime.combiner.class. Combiner will not be used
2016-01-14 18:31:47,911 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(182)) - Shuffle assigned with 1 inputs, codec: NoneifileReadAhead: true
2016-01-14 18:31:47,910 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(289)) - Num IOs determined for AutoStart: 1
2016-01-14 18:31:47,911 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:<init>(158)) - ShuffleScheduler running for sourceVertex: C4C29BDD94924BE781EC5BE50BF8EFCE with configuration: maxFetchFailuresBeforeReporting=5, reportReadErrorImmediately=true, maxFailedUniqueFetches=1, abortFailureLimit=30, maxMapRuntime=0, maxTaskOutputAtOnce=20
2016-01-14 18:31:47,911 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(221)) - InitialRequest: ShuffleMem=634178752, postMergeMem=0, RuntimeTotalAvailable=472110858. Updated to: ShuffleMem=472110858, postMergeMem: 0
2016-01-14 18:31:47,911 INFO orderedgrouped.MergeManager (MergeManager.java:<init>(252)) - MergerManager: memoryLimit=472110858, maxSingleShuffleLimit=118027712, mergeThreshold=424899744, ioSortFactor=100, memToMemMergeOutputsThreshold=100
2016-01-14 18:31:47,911 INFO orderedgrouped.Shuffle (Shuffle.java:<init>(233)) - Num fetchers being started: 1
2016-01-14 18:31:47,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(291)) - Waiting for 1 IOs to start
2016-01-14 18:31:47,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:call(421)) - Started Input with src edge: C4C29BDD94924BE781EC5BE50BF8EFCE
2016-01-14 18:31:47,912 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:initialize(304)) - AutoStartComplete
2016-01-14 18:31:47,912 INFO task.TezTaskRunner (TezTaskRunner.java:run(175)) - Running task, taskAttemptId=attempt_1452796307645_0001_1_01_000000_0
2016-01-14 18:31:47,913 INFO orderedgrouped.ShuffleInputEventHandlerOrderedGrouped (ShuffleInputEventHandlerOrderedGrouped.java:processDataMovementEvent(84)) - DME srcIdx: 0, targetIdx: 0, attemptNum: 0, payload: [host: ip-10-187-41-40, port: 0, pathComponent: attempt_1452796307645_0001_1_00_000000_0_10339, runDuration: 44488, ]
2016-01-14 18:31:47,914 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logIndividualFetchComplete(250)) - Completed fetch for attempt: InputAttemptIdentifier [inputIdentifier=InputIdentifier [inputIndex=0], attemptNumber=0, pathComponent=attempt_1452796307645_0001_1_00_000000_0_10339] to DISK_DIRECT, CompressedSize=301, DecompressedSize=297,EndTime=1452796307914, TimeTaken=1, Rate=0.29 MB/s
2016-01-14 18:31:47,914 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:copySucceeded(213)) - All inputs fetched for input vertex : C4C29BDD94924BE781EC5BE50BF8EFCE
2016-01-14 18:31:47,914 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:logProgress(263)) - copy(1 of 1. Transfer rate (CumulativeDataFetched/TimeSinceInputStarted)) 0.00 MB/s)
2016-01-14 18:31:47,914 INFO orderedgrouped.ShuffleScheduler (ShuffleScheduler.java:freeHost(529)) - ip-10-187-41-40:0 freed by fetcher [C4C29BDD94924BE781EC5BE50BF8EFCE] #57 in 1ms
2016-01-14 18:31:47,915 INFO orderedgrouped.Shuffle (Shuffle.java:cleanupFetchers(394)) - Shutdown..fetcher [C4C29BDD94924BE781EC5BE50BF8EFCE] #57, status:false, isInterrupted:false
2016-01-14 18:31:47,915 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 1 on-disk map-outputs
2016-01-14 18:31:47,915 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 1 files, 301 bytes from disk
2016-01-14 18:31:47,915 INFO graph.Hadoop2TezStreamGraph (Hadoop2TezStreamGraph.java:buildGraph(151)) - using streamed source: GroupBy(every)[by:[{?}:ALL]]
2016-01-14 18:31:47,916 INFO tez.FlowProcessor (FlowProcessor.java:run(121)) - sourcing from: GroupBy(every)[by:[{?}:ALL]] streamed: true, id: 9BF40797CC364FF488F6D0231D0C6313
2016-01-14 18:31:47,916 INFO tez.FlowProcessor (FlowProcessor.java:run(124)) - sinking to: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP"], id: 3831E03A344D48F6A1078CE41C53AD00
2016-01-14 18:31:47,916 INFO element.TezSinkStage (TezSinkStage.java:prepare(57)) - calling MROutput#start() on: Hfs["TextLine[['offset', 'line']->[ALL]]"]["/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output/hadoop2-tez/everypipeassemblyplatform/ALL_ARGS_SWAP"]
2016-01-14 18:31:47,916 INFO element.TezGroupGate (TezGroupGate.java:prepare(88)) - calling OrderedGroupedKVInput#start() on: GroupBy(every)[by:[{?}:ALL]] 9BF40797CC364FF488F6D0231D0C6313, for 1 inputs
2016-01-14 18:31:47,916 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:47,917 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 1 sorted segments
2016-01-14 18:31:47,917 INFO impl.TezMerger (TezMerger.java:merge(665)) - Down to the last merge-pass, with 1 segments left of total size: 282 bytes
2016-01-14 18:31:47,917 INFO orderedgrouped.Shuffle (Shuffle.java:call(381)) - merge complete for input vertex : C4C29BDD94924BE781EC5BE50BF8EFCE
2016-01-14 18:31:47,917 INFO orderedgrouped.Shuffle (Shuffle.java:onSuccess(488)) - Shuffle Runner thread complete
2016-01-14 18:31:47,917 INFO tez.FlowProcessor (FlowProcessor.java:waitForInputsReady(202)) - flow node id: D8DC787A9EA3491D912A140740897384, all 1 inputs ready in: 00:00:00.000
2016-01-14 18:31:47,918 INFO common.ValuesIterator (ValuesIterator.java:<init>(83)) - keyDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@6856e313; valueDeserializer=cascading.tuple.hadoop.io.TupleDeserializer@205629e9
2016-01-14 18:31:47,918 ERROR element.TrapHandler (TrapHandler.java:handleReThrowableException(115)) - caught Throwable, no trap available, rethrowing
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:47,921 ERROR element.TezGroupByGate (TezGroupByGate.java:reduce(92)) - caught throwable
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:47,922 INFO output.MROutput (MROutput.java:flush(534)) - Flushing Simple Output
2016-01-14 18:31:47,922 INFO output.MROutput (MROutput.java:flush(544)) - Flushed Simple Output
2016-01-14 18:31:47,923 INFO app.TaskAttemptListenerImpTezDag (TaskAttemptListenerImpTezDag.java:canCommit(353)) - Commit go/no-go request from attempt_1452796307645_0001_1_01_000000_0
2016-01-14 18:31:47,923 INFO impl.TaskImpl (TaskImpl.java:canCommit(750)) - attempt_1452796307645_0001_1_01_000000_0 given a go for committing the task output.
2016-01-14 18:31:47,924 INFO task.TezTaskRunner (TezTaskRunner.java:run(221)) - Encounted an error while executing task: attempt_1452796307645_0001_1_01_000000_0
cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-01-14 18:31:47,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_ATTEMPT_FINISHED to history file
2016-01-14 18:31:47,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:TASK_ATTEMPT_FINISHED]: vertexName=D8DC787A9EA3491D912A140740897384, taskAttemptId=attempt_1452796307645_0001_1_01_000000_0, startTime=1452796307889, finishTime=1452796307926, timeTaken=37, status=FAILED, errorEnum=FRAMEWORK_ERROR, diagnostics=Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
, counters=Counters: 43, File System Counters, FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2, Shuffle Errors, BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796307918, Process_Duration=6, Process_End_Time=1452796307924, Read_Duration=0, Tuples_Read=1
2016-01-14 18:31:47,926 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307645_0001_1_01_000000_0 TaskAttempt Transitioned from RUNNING to FAIL_IN_PROGRESS due to event TA_FAILED
2016-01-14 18:31:47,926 INFO impl.TaskImpl (TaskImpl.java:transition(1296)) - Failing task: task_1452796307645_0001_1_01_000000, currentFailedAttempts: 1, maxFailedAttempts: 1
2016-01-14 18:31:47,926 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event TASK_FINISHED to history file
2016-01-14 18:31:47,926 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:TASK_FINISHED]: vertexName=D8DC787A9EA3491D912A140740897384, taskId=task_1452796307645_0001_1_01_000000, startTime=1452796307889, finishTime=1452796307926, timeTaken=37, status=FAILED, successfulAttemptID=null, diagnostics=TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
], counters=Counters: 0
2016-01-14 18:31:47,927 INFO impl.TaskImpl (TaskImpl.java:handle(862)) - task_1452796307645_0001_1_01_000000 Task Transitioned from RUNNING to FAILED due to event T_ATTEMPT_FAILED
2016-01-14 18:31:47,927 INFO impl.VertexImpl (VertexImpl.java:transition(3536)) - Num completed Tasks for vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] : 1
2016-01-14 18:31:47,927 INFO impl.VertexImpl (VertexImpl.java:transition(3543)) - Failing vertex: vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] because task failed: task_1452796307645_0001_1_01_000000
2016-01-14 18:31:47,927 INFO impl.VertexImpl (VertexImpl.java:tryEnactKill(1842)) - Killing tasks in vertex: vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] due to trigger: OWN_TASK_FAILURE
2016-01-14 18:31:47,927 INFO impl.VertexImpl (VertexImpl.java:checkVertexForCompletion(1754)) - Vertex failed as one or more tasks failed. failedTasks:1
2016-01-14 18:31:47,927 INFO impl.VertexImpl (VertexImpl.java:abortVertex(3302)) - Invoking committer abort for vertex, vertexId=vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384]
2016-01-14 18:31:47,927 INFO impl.VertexImpl (VertexImpl.java:run(3310)) - Invoking committer abort for output=3831E03A344D48F6A1078CE41C53AD00, vertexId=vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384]
2016-01-14 18:31:47,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(712)) - Final Counters : Counters: 43 [[File System Counters FILE_BYTES_READ=333, FILE_BYTES_WRITTEN=8, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0][org.apache.tez.common.counters.TaskCounter REDUCE_INPUT_GROUPS=0, REDUCE_INPUT_RECORDS=1, COMBINE_INPUT_RECORDS=0, SPILLED_RECORDS=0, NUM_SHUFFLED_INPUTS=1, NUM_SKIPPED_INPUTS=0, NUM_FAILED_SHUFFLE_INPUTS=0, MERGED_MAP_OUTPUTS=1, GC_TIME_MILLIS=0, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, OUTPUT_RECORDS=0, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=4, SHUFFLE_BYTES=301, SHUFFLE_BYTES_DECOMPRESSED=297, SHUFFLE_BYTES_TO_MEM=0, SHUFFLE_BYTES_TO_DISK=0, SHUFFLE_BYTES_DISK_DIRECT=301, NUM_MEM_TO_DISK_MERGES=0, NUM_DISK_TO_DISK_MERGES=0, SHUFFLE_PHASE_TIME=4, MERGE_PHASE_TIME=6, FIRST_EVENT_RECEIVED=2, LAST_EVENT_RECEIVED=2][Shuffle Errors BAD_ID=0, CONNECTION=0, IO_ERROR=0, WRONG_LENGTH=0, WRONG_MAP=0, WRONG_REDUCE=0][cascading.flow.SliceCounters Process_Begin_Time=1452796307918, Process_Duration=6, Process_End_Time=1452796307924, Read_Duration=0, Tuples_Read=1]]
2016-01-14 18:31:47,928 INFO tez.FlowProcessor (LogUtil.java:logMemory(59)) - flow node id: D8DC787A9EA3491D912A140740897384, mem on close (mb), free: 266, total: 565, max: 672
2016-01-14 18:31:47,928 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(729)) - Closed processor for vertex=D8DC787A9EA3491D912A140740897384, index=1
2016-01-14 18:31:47,929 INFO orderedgrouped.Shuffle (Shuffle.java:shutdown(320)) - Shutting down Shuffle for source: C4C29BDD94924BE781EC5BE50BF8EFCE
2016-01-14 18:31:47,929 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:handleEvent(149)) - Processing the event EventType: S_TA_ENDED
2016-01-14 18:31:47,929 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(854)) - finalMerge called with 0 in-memory map-outputs and 0 on-disk map-outputs
2016-01-14 18:31:47,929 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:VERTEX_FINISHED]: vertexName=D8DC787A9EA3491D912A140740897384, vertexId=vertex_1452796307645_0001_1_01, initRequestedTime=1452796307806, initedTime=1452796307813, startRequestedTime=1452796307818, startedTime=1452796307818, finishTime=1452796307928, timeTaken=110, status=FAILED, diagnostics=Task failed, taskId=task_1452796307645_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]]
Vertex failed as one or more tasks failed. failedTasks:1
Vertex vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] killed/failed due to:null, counters=Counters: 0, vertexStats=firstTaskStartTime=1452796307889, firstTasksToStart=[ task_1452796307645_0001_1_01_000000 ], lastTaskFinishTime=1452796307926, lastTasksToFinish=[ task_1452796307645_0001_1_01_000000 ], minTaskDuration=-1, maxTaskDuration=-1, avgTaskDuration=-1.0, numSuccessfulTasks=0, shortestDurationTasks=[ ], longestDurationTasks=[ ], vertexTaskStats={numFailedTasks=1, numSucceededTasks=0, numKilledTaskAttempts=0, numKilledTasks=0, numFailedTaskAttempts=1, numCompletedTasks=1}
2016-01-14 18:31:47,929 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event VERTEX_FINISHED to history file
2016-01-14 18:31:47,929 INFO impl.VertexImpl (VertexImpl.java:handle(1564)) - vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] transitioned from RUNNING to FAILED due to event V_TASK_COMPLETED
2016-01-14 18:31:47,929 INFO impl.DAGImpl (DAGImpl.java:transition(1747)) - Vertex vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] completed., numCompletedVertices=2, numSuccessfulVertices=1, numFailedVertices=1, numKilledVertices=0, numVertices=2
2016-01-14 18:31:47,929 INFO impl.DAGImpl (DAGImpl.java:checkDAGForCompletion(1082)) - DAG failed due to vertex failure. failedVertices:1 killedVertices:0
2016-01-14 18:31:47,929 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(831)) - Calling DAG commit/abort for dag: dag_1452796307645_0001_1
2016-01-14 18:31:47,930 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(917)) - No output committers for vertex: vertex_1452796307645_0001_1_00 [C4C29BDD94924BE781EC5BE50BF8EFCE]
2016-01-14 18:31:47,930 INFO impl.DAGImpl (DAGImpl.java:commitOrAbortOutputs(926)) - Aborting output: 3831E03A344D48F6A1078CE41C53AD00 for vertex: vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384]
2016-01-14 18:31:47,930 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(951)) - Merging 0 files, 0 bytes from disk
2016-01-14 18:31:47,930 INFO orderedgrouped.MergeManager (MergeManager.java:finalMerge(966)) - Merging 0 segments, 0 bytes from memory into reduce
2016-01-14 18:31:47,930 INFO impl.TezMerger (TezMerger.java:merge(560)) - Merging 0 sorted segments
2016-01-14 18:31:47,931 INFO impl.TezMerger (TezMerger.java:merge(562)) - Nothing to merge. Returning an empty iterator
2016-01-14 18:31:47,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(753)) - Close input for vertex=D8DC787A9EA3491D912A140740897384, sourceVertex=C4C29BDD94924BE781EC5BE50BF8EFCE
2016-01-14 18:31:47,931 INFO recovery.RecoveryService (RecoveryService.java:handle(306)) - DAG completed, dagId=dag_1452796307645_0001_1, queueSize=0
2016-01-14 18:31:47,931 INFO runtime.LogicalIOProcessorRuntimeTask (LogicalIOProcessorRuntimeTask.java:cleanup(772)) - Close input for vertex=D8DC787A9EA3491D912A140740897384, sourceVertex=3831E03A344D48F6A1078CE41C53AD00
2016-01-14 18:31:47,931 INFO task.TezChild (TezChild.java:shutdown(343)) - Shutdown invoked for container container_1452796307645_0001_00_000002
2016-01-14 18:31:47,932 INFO task.TezChild (TezChild.java:shutdown(345)) - Shutting down container container_1452796307645_0001_00_000002
2016-01-14 18:31:47,932 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(270)) - ContainerExecutionResult for: container_1452796307645_0001_00_000002 = ContainerExecutionResult{exitStatus=EXECUTION_FAILURE, throwable=org.apache.tez.dag.api.TezException: java.lang.reflect.UndeclaredThrowableException, errorMessage='TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException'}
2016-01-14 18:31:47,932 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:onSuccess(279)) - Container: container_1452796307645_0001_00_000002 completed but with errors
2016-01-14 18:31:47,931 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:DAG_FINISHED]: dagId=dag_1452796307645_0001_1, startTime=1452796307804, finishTime=1452796307930, timeTaken=126, status=FAILED, diagnostics=Vertex failed, vertexName=D8DC787A9EA3491D912A140740897384, vertexId=vertex_1452796307645_0001_1_01, diagnostics=[Task failed, taskId=task_1452796307645_0001_1_01_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Failure while running task:cascading.tuple.TupleException: operation added the wrong number of fields, expected: ['offset', 'line'], got result size: 1
at cascading.tuple.TupleEntryCollector.add(TupleEntryCollector.java:125)
at cascading.TestAggregator.complete(TestAggregator.java:96)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:153)
at cascading.flow.stream.element.AggregatorEveryStage.completeGroup(AggregatorEveryStage.java:41)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:51)
at cascading.flow.stream.duct.OpenReducingDuct.receive(OpenReducingDuct.java:28)
at cascading.flow.hadoop.stream.HadoopGroupGate.accept(HadoopGroupGate.java:141)
at cascading.flow.tez.stream.element.TezGroupByGate.reduce(TezGroupByGate.java:84)
at cascading.flow.tez.stream.element.TezGroupGate.run(TezGroupGate.java:112)
at cascading.flow.tez.FlowProcessor.run(FlowProcessor.java:165)
at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:334)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:176)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:168)
at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.call(TezTaskRunner.java:163)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
]], Vertex failed as one or more tasks failed. failedTasks:1, Vertex vertex_1452796307645_0001_1_01 [D8DC787A9EA3491D912A140740897384] killed/failed due to:null]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0, counters=Counters: 31, org.apache.tez.common.counters.DAGCounter, NUM_FAILED_TASKS=1, NUM_SUCCEEDED_TASKS=1, TOTAL_LAUNCHED_TASKS=2, RACK_LOCAL_TASKS=1, File System Counters, FILE_BYTES_READ=109, FILE_BYTES_WRITTEN=333, FILE_READ_OPS=0, FILE_LARGE_READ_OPS=0, FILE_WRITE_OPS=0, org.apache.tez.common.counters.TaskCounter, SPILLED_RECORDS=20, GC_TIME_MILLIS=7, CPU_MILLISECONDS=0, PHYSICAL_MEMORY_BYTES=0, VIRTUAL_MEMORY_BYTES=0, COMMITTED_HEAP_BYTES=592445440, INPUT_RECORDS_PROCESSED=0, OUTPUT_RECORDS=20, OUTPUT_BYTES=251, OUTPUT_BYTES_WITH_OVERHEAD=297, OUTPUT_BYTES_PHYSICAL=301, ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0, ADDITIONAL_SPILL_COUNT=0, cascading.flow.SliceCounters, Process_Begin_Time=1452796307873, Process_Duration=2, Process_End_Time=1452796307875, Read_Duration=0, Tuples_Read=20, Tuples_Written=20, Write_Duration=2, cascading.flow.StepCounters, Tuples_Read=20
2016-01-14 18:31:47,931 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event DAG_FINISHED to history file
2016-01-14 18:31:47,932 INFO impl.DAGImpl (DAGImpl.java:finished(1180)) - DAG: dag_1452796307645_0001_1 finished with state: FAILED
2016-01-14 18:31:47,933 INFO impl.DAGImpl (DAGImpl.java:handle(973)) - dag_1452796307645_0001_1 transitioned from RUNNING to FAILED due to event DAG_VERTEX_COMPLETED
2016-01-14 18:31:47,933 INFO history.HistoryEventHandler (HistoryEventHandler.java:handleCriticalEvent(110)) - [HISTORY][DAG:dag_1452796307645_0001_1][Event:CONTAINER_STOPPED]: containerId=container_1452796307645_0001_00_000002, stoppedTime=1452796307933, exitStatus=0
2016-01-14 18:31:47,933 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:handleEvent(151)) - Writing event CONTAINER_STOPPED to history file
2016-01-14 18:31:47,934 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000002 transitioned from RUNNING to STOP_REQUESTED via event C_STOP_REQUEST
2016-01-14 18:31:47,934 INFO node.AMNodeImpl (AMNodeImpl.java:transition(294)) - Attempt failed on node: 127.0.0.1:0 TA: attempt_1452796307645_0001_1_01_000000_0 failed: true container: container_1452796307645_0001_00_000002 numFailedTAs: 0
2016-01-14 18:31:47,935 INFO container.AMContainerImpl (AMContainerImpl.java:transition(1033)) - Container container_1452796307645_0001_00_000002 exited with diagnostics set to TaskExecutionFailure: java.lang.reflect.UndeclaredThrowableException
2016-01-14 18:31:47,935 INFO container.AMContainerImpl (AMContainerImpl.java:handle(442)) - AMContainer container_1452796307645_0001_00_000002 transitioned from STOP_REQUESTED to COMPLETED via event C_COMPLETED
2016-01-14 18:31:47 Completed Dag: dag_1452796307645_0001_1
2016-01-14 18:31:47,936 INFO app.DAGAppMaster (DAGAppMaster.java:handle(579)) - DAG completed, dagId=dag_1452796307645_0001_1, dagState=FAILED
2016-01-14 18:31:47,936 INFO common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(164)) - Redirecting log file based on addend: dag_1452796307645_0001_1_post
2016-01-14 18:31:47,936 WARN common.TezUtilsInternal (TezUtilsInternal.java:updateLoggers(179)) - Not configured with appender named: CLA. Cannot reconfigure logger output
2016-01-14 18:31:47,936 INFO app.DAGAppMaster (DAGAppMaster.java:handle(625)) - Waiting for next DAG to be submitted.
2016-01-14 18:31:47,936 INFO launcher.LocalContainerLauncher (LocalContainerLauncher.java:stop(242)) - Ignoring stop request for containerId: container_1452796307645_0001_00_000002
2016-01-14 18:31:47,936 INFO impl.TaskAttemptImpl (TaskAttemptImpl.java:handle(692)) - attempt_1452796307645_0001_1_01_000000_0 TaskAttempt Transitioned from FAIL_IN_PROGRESS to FAILED due to event TA_CONTAINER_TERMINATED
2016-01-14 18:31:47,937 INFO client.TezClient (TezClient.java:stop(495)) - Shutting down Tez Session, sessionName=(1/1) ...blyplatform/ALL_ARGS_SWAP, applicationId=application_1452796307645_0001
2016-01-14 18:31:47,938 INFO client.DAGClientHandler (DAGClientHandler.java:shutdownAM(117)) - Received message to shutdown AM
2016-01-14 18:31:47,938 INFO rm.TaskSchedulerEventHandler (TaskSchedulerEventHandler.java:setShouldUnregisterFlag(580)) - TaskScheduler notified that it should unregister from RM
2016-01-14 18:31:47,938 INFO app.DAGAppMaster (DAGAppMaster.java:shutdownTezAM(1088)) - No current running DAG, shutting down the AM
2016-01-14 18:31:47,938 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(692)) - DAGAppMasterShutdownHandler invoked
2016-01-14 18:31:47,938 INFO app.DAGAppMaster (DAGAppMaster.java:shutdown(701)) - Handling DAGAppMaster shutdown
2016-01-14 18:31:47,939 INFO app.DAGAppMaster (DAGAppMaster.java:run(721)) - Sleeping for 5 seconds before shutting down
2016-01-14 18:31:47,939 INFO history.HistoryEventHandler (HistoryEventHandler.java:serviceStop(79)) - Stopping HistoryEventHandler
2016-01-14 18:31:47,939 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(189)) - Stopping RecoveryService
2016-01-14 18:31:47,939 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(192)) - Handle the remaining events in queue, queue size=0
2016-01-14 18:31:47,939 INFO recovery.RecoveryService (RecoveryService.java:run(166)) - EventQueue take interrupted. Returning
2016-01-14 18:31:47,939 INFO recovery.RecoveryService (RecoveryService.java:serviceStop(214)) - Closing Summary Stream
2016-01-14 18:31:47,940 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:serviceStop(118)) - Stopping SimpleHistoryLoggingService, eventQueueBacklog=0
2016-01-14 18:31:47,940 INFO impl.SimpleHistoryLoggingService (SimpleHistoryLoggingService.java:run(105)) - EventQueue take interrupted. Returning
2016-01-14 18:31:47,943 INFO app.DAGAppMaster (DAGAppMaster.java:run(1773)) - Completed deletion of tez scratch data dir, path=file:/opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/tmp/.staging/396C353FD4834B02B6F6E8C678759F25/CB0129970503467B8054F4C0EED81510/.tez/application_1452796307645_0001
2016-01-14 18:31:47,943 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping all jobs
2016-01-14 18:31:47,944 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopping: (1/1) ...blyplatform/ALL_ARGS_SWAP
2016-01-14 18:31:47,944 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] stopped all jobs
2016-01-14 18:31:47,944 INFO flow.Flow (BaseFlow.java:logInfo(1433)) - [every] completed in: 00:00.317
2016-01-14 18:31:47,944 INFO cascading.PlatformTestCase (PlatformTestCase.java:tearDown(189)) - copying to local /opt/buildAgent-8.0.3/work/ca57560bc31094a7/apache-2.6.x-tez-0.6.x/build/test/output//hadoop2-tez/everypipeassemblyplatform//ALL_ARGS_SWAP